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ABSTRACT 


The increased use of simulation games in education has prompted 
concern about the measurement of student performance within such a setting, 
Tansey (1970) has stated, "What is now urgently required is research... 
with finding evaluative systems that will enable the worth of the basic 
Technique to be measured", 

The present study investigated the efficacy of one possible 
analytic system - Tucker's (1960, 1966) component curve analysis. The m- 
thod was originally designed for analyzing a series of scores on a learn- 
ing task for each of a number of individuals, and thus appears suitable 
for analyzing data from simulation games that provide some form of score 
at the end of each trial. 

The sample consisted of 67 grade eight students that were ran- 
domly selected from one junior high school in Edmonton, Alberta. Each 
student interacted with a computer-based simulation game on the salmon 
fishing industry in British Columbia for a period of 10 cycles or 'years', 
The scores for each student consisted of the value of the "cash on hand" 
variable at the end of each cycle. The students then received a computer- 
based version of the Matching Familiar Figures test, designed to provide 
a measure of impulsivity. The following week a set of 10 marker tests 
from the Kit of Reference Tests for Cognitive Factors (French, Ekstrom, 
and Price, 1967) were administered. 

A component curve analysis of the raw performance data provided 
4 component curves (i.e. learning curves) that were representative of 4 
different patterns of performance. Factor scores for the students on the 


four curves were computed, Approximately one quarter of the students were 
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identified with a single curve, Orthogonal rotations in the person space 
increased this proportion to one half, 

Additional component curve analyses were performed on the data 
after (a) the trial means were removed, (b) the person means were removed, 
and (c) both trial and person means were removed. There were no major 
differences among the results. 

The measures obtained from the cognitive tests Bnd the Matching 
Familiar Figures test were factor analyzed using Joreskogs (1967) maximum 
likelihood algorithm and then rotated using an equamax criterion, Five 
factors were identified: Associative (Rote) Memory, Verbal Compreshension, 
Number Facility, General Reasoning, and Impulsivity. 

The relationship between the set of cognitive factor scores and 
the scores on the component curves was examined by means of canonical 
correlation and multiple linear regression. The main relationship was be- 
tween the first component curve and the cognitive factors of Impulsivity, 
General Reasoning, and Number Facility. Additional relationships were 
observed between scores on component curve II and scores on General Rea- 
soning and Impulsivity, and between curve IV and Number Facility. The 
three solutions obtained from the data matrices with trial and/or person 
means removed did not exhibit as strong a relationship with the cognitive 
factors, In simulation tasks of this type the use of the raw data matrix, 
which retains all of the available information, was recommended as the 
most satisfactory of the four alternatives for input to the component 
curve analysis. 

A set of "random" performance data was generated by means of 
Monte Carlo techniques and then analyzed by means of Tucker's component 


curve analysis, Three component curves were required to account for the 
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data. Although this was unexpected, it may imply an additional applica- 
tion for Tucker's method since the number and shape of the component 
curves resulting from an analysis of random performance could be used as 
a method of classification for simulation games, 

Referring to the component curve analysis of the actual student 
performance data, individual differences in performance were shown to be 
related to individual differences in cognitive abilities. The results of 
the study have indicated the potential value of Tucker's component curve 


analysis for describing student performance on a complex simulation game. 
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CHAPTER I 


INTRODUCTION 


I. PURPOSE OF THE STUDY 


The primary purpose of this study is to evaluate Tucker's com- 
ponent curve analysis as a method for describing student performance on 
a computer-based simulation game. Component curve analysis was designed 
to evaluate a series of scores on a learning task for each of a number 
of individuals. The method seems particularly appropriate for examining 
student performance on a simulation game that involves repetitive deci- 
Sion making with a "score" computed at the end of each set of decisions. 
Thus the study contains two main areas of interest: the context within 
which the data were obtained, computer-based simulation gaming, and the 
method of analyzing such data, component curve analysis. 

The principal focal point is one of measurement - to further 
the understanding of the nature of the results provided by component curve 
analysis. The efficiency of the method, considered as a gain resulting 
from a reduction of data versus an accompanying loss of precision in re- 
producing the observed scores will be evaluated by means of the following 
criteria: the number of component curves compared to the number of trials, 
the shape of the component curves, the meaningfulness of various rotations, 
and the relationship of the factor scores for each component curve with a 
set of external measures (cognitive factors). Canonical correlation and 
multiple linear regression will be used to describe this latter relation- 


ship. The importance of evaluating a method for measuring student 
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2 


performance on a computer-based simulation game is discussed in the next 


section, which provides a brief history of the development of this new 
instructional strategy within the context of the concurrent development 


in computer technology. 


II. IMPORTANCE OF THE STUDY 


The full impact of the new computer technology on our society 
is difficult to envision. In 1946 the first electronic computer, ENIAC, 
became operational. Today the variety of computers presently being used 
ranges from small desk-top stored program calculators to large time- 
sharing modular systems capable of handling the needs of thousands of 
users. 

Education must be included in the list of professions that are 
likely to undergo major transformations in order to realize the potential 
provided. At present the impact on education may be felt in two areas: 

1, Data processing applications in school administration, 

2. Computational power needed for education research, 

However as experience leads to refinements in these two areas, new areas 
will also develop. Two such new areas that appear to have a large poten- 
tial are computer-assisted instruction (CAI) and real-time information 
retrieval systems, 

Within the area of computer-assisted instruction there are at 
least three advantages provided by computer technology. First, complex 
calculations may be carried out in very short periods of time. This may 


be particularly relevant in subject areas such as mathematics, physics, 
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chemistry, and economics. Second, the use of graphics on automatic dis- 
play devices will provide a pictorial representation of complex situa- 
tions which in turn may lead to new problem-solving behaviors. Both of 
these advantages utilize the power of the computer to extend man's own 
capabilities. An example that utilizes both of these advantages is des- 
cribed by Levinthal (1966). He discusses a computer program that con- 
structs mathematical models of large complex biological molecules and 
controls an equivalent pictorial display on an oscilloscope screen. The 
third advantage involves the possibility of utilizing the characteristics 
of present learning behavior to modify, and hopefully to improve, the 
characteristics of future learning behavior. 

One type of instructional strategy that may be included under 
the heading of computer-assisted instruction is called simulation. The 
simulation of many situations that may or will occur in a person's real 
experience, or simulation of situations as a means or technique of teach- 
ing a particular concept has not been feasible until recently. However 
with the advent of computer technology such approaches will now become 
practical. 

The advantages of computer-based simulation have been enumer- 
ated by a number of authors. Thus Bushnell (1967) feels that computer 
simulation possesses advantages over the actual observations of the na- 
tural events themselves. 

Time may be speeded up or slowed down. In comparison with 

traditional instruction through lectures and text-books, si- 

mulation brings a sense of immediacy to the learning task and 
may be considerably more realistic--thereby challenging the 
student to participate more actively. The student may learn 


to deal with systems far more complex than any he could learn 
to describe accurately in the same amount of time (p. 61). 
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Similarly Koch (1968) mentions the following advantages: enjoyment, hold- 
ing the attention for a prolonged period, tendency for emotional involve- 
ment, avoidance of costly consequences of erroneous decisions, providing 
practice that could be too costly or unavailable otherwise, ability to 
add controlled feedback, progression from simple to complex, and ee 
nation of negative or harmful experience. Also the subjects are able to 
see the consequences of their actions and to understand the interrelation- 
ships in the simulation. Carlson (p. 167-174) reviews the opinions of a 
number of researchers on the value of games as an instructional technique. 
Whether or not such an approach succeeds in its intended goal 
may depend upon a number of variables. It may be that only students with 
a certain range of characteristics will benefit from such an approach, 
Analogously, it may be that there are characteristics of the simulation 
environment that will either enhance or reduce the possibility of success, 
Stolurow (1965), for example, feels that the essentials of programmed in- 
struction fall under the following three headings--what is to be taught, 
who will be taught (i.e. characteristics of students), and how it will be 
taught (i.e. the teaching strategy). Furthermore, it is natural to hypo- 
thesize an interaction effect between these variables. Thus Bundy (1968) 
feels that ways must be found to design programs which will be able, ina 
sophisticated way, to analyze and act on student background, ability, and 
progress. This implies a systematic classification of individual diffe- 
rence variables and the study of their interaction with instructional 
materials and modes of presentation. 
Further knowledge of these variables will enable a course plan- 


ner to maximize the benefit achieved by including different modes of 
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instruction in a CAI environment. Within the restricted context of simu- 
lation, the instructor should be able to make use of the student's cha- 
racteristics and performance in determining a suitable type of simulation 
model most appropriate for efficiently teaching a specified concept. 
Implicit in the above statements is the assumption that power- 
ful methods uf analyzing student performance are presently available. Be- 
cause of the complexity of the complete situation a multivariate approach 
would appear to merit special attention. One such development, referred 
to as component curve analysis, has been outlined by Tucker (1960, 1966). 
However Tucker's approach has only been applied to relatively simple 
learning situations - that of probability learning tasks. The present 
study examines the suitability of this approach for a much more complex 
learning situation, that of student performance during a computer-based 


Simulation exercise, 


III. SCOPE OF THE STUDY 

Edwards and Cronbach (1952) distinguish two types of research 
- survey research and critical research. Thus survey research is under- 
taken when we are relatively ignorant of possible relationships among 
the variables and when we may even lack knowledge concerning the perti- 
nent variables. With critical research theoretical considerations indi- 
cate the questions to be asked and even indicate the answers to be expec- 
ted. Using Cronbach's definition as given above, the present study may 
be classified as survey research. 

In order to meaningfully examine relationships between students 


and simulation models, it is important that the simulation model be 
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adequately described. At present there are no universal standards for 
describing a simulation model. However if comparisons are to be made 
among different studies using different simulation models, it is essen- 
tial that descriptions of the models be provided. Therefore the first 
objective of this study is to define the dimensions for describing the 
Simulation tasks used in the present investigation. 

The second objective is to measure learning during the entire 
simulation process. This is much more desirable than using methods that 
only select and measure performance at one or two specified points, usu- 
ally the last trial. Tucker's component curve analysis represents one 
such approach that utilizes information from the whole exercise and the 
efficacy of this method in studying learning under a complex repetitive 
task is evaluated. 

The third objective of this study is to examine the relations 
between learning as measured by the component curves and outside measures 
of individual abilities and ayhenueneaee that have been shown in other 
studies to be related to learning tasks. 

Therefore the results of this study may be summarized as follows: 

1. to establish dimensions for use in describing simulation 

exercises 

2. to evaluate Tucker's component curve analysis for measuring 

learning in a complex task 

3. to relate performance in a simulation task as measured by 


component curves to a set of external measures. 
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CHAPTER II 
SOME RELATED THEORY AND RESEARCH 


I. COMPONENT CURVE ANALYSIS 


Eckart -Young Theorem 


Eckart and Young (1936) have developed a method for approxima- 
ting, in a least squares sense, any matrix X by a matrix X of lower rank. 


The Eckart-Young analysis of the data matrix X is given by the 


equation: 
n'N 7 ran in nin 

where V is an orthogonal matrix (V! = v4) 
W is an orthogonal matrix (W' = wy 


G contains the eigenvalues, a. in the diagonal elements of 
the matrix and zeros elsewhere. 
Tucker's component curve analysis uses a variant of this procedure to 
analyze data from a learning task. When the scores in the X matrix re- 
present learning data, then the V matrix reflects a set of learning curves 
and the W matrix describes the Pagtduai*s behavior relative to these 


curves. 


Tucker's Component Curve Analysis 


The essential feature of component curve analysis resides in 
the type of data which forms the matrix X that undergoes the analysis. 
Tucker's method consists of applying the Eckart-Young theorem to a par- 


ticular type of data matrix. The raw data must consist of a series of 
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scores on some task for each of a number of individuals - essentially a 
repeated measures design applied to some learning situation. 

Tucker (1960, 1966) has shown the suitableness of the component 
curve analysis for studying the learning curves for a probability learn- 
ing task. Tucker used the following model: 


= +t: a ° ° e 
arr Maes Pot os 


where ae is the score on the trial j (j = 1, 2, .. .m) for individual 
Pere oes Ty Ie Dio? etc. are coefficients dependent on the 


Eriais, and Y ¥ i? etc. are individual parameters. There are m trials 


Lien vie 
and n individuals. This model, which is identical with the basic factor 
analytic model, may be interpreted as follows, "the series of scores of 
an individual is a weighted sum of reference learning curves, the weights 
being the individual factor scores and the series of b's for each factor 
forming a reference learning curve (p. 483)." 

Using the Eckart-Young theorem, an approximate matrix X, (of 


rank k) is formed by using the first k left principal vectors, (cols.) in 


V, principal roots in G, and right principal vectors (rows) of W to form 


the matrices Vo Gis We such that 
A 
ay G W. 
ae n Ke k Ke k Sy (1) 


Tucker defines the matrices, 


2 d i fol 
By an YE as follows: 
eh -Li2 
BL = N VF, (2) 
* yz 
tee =a Wy, (3) 
where nth? is a scaling factor to ensure that the mean square of each row 
* 
of Yy is unity. 
A * * 
Thus Ry peat (4) 
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Recall that X = VGW 
and therefore XX' = VGW'G!y! 
VGG'yv! 


ve7v' 


This equation can be solved using the familiar Hotelling prin- 
cipal axes algorithm. Jacobi and Householder have developed two alter- 
nate methods for solving this equation. The number of principal roots 
and vectors may be determined by inspection and By is easily computed 


* 
using equation (2). It remains to show how the values for Y. are deter- 


k 
mined. 
Si BS y = X 
ince ko = XR» 
*1 
pre-multiplying both sides by By yields 
pee an eae (5) 
pene ee eeepc kis ? 
Al Be = yf? V,G ( tion'(2) ) 
so ete! KEK see equation . 
Therefore 
NS pect Ao ee ees 9 By, : 
BE = N Gy Vie = N G. Vy ; 


Substitution into equation (5) gives 


1/2) ee =1f2 ce aes 
(N GV) (N Mie) SY co SB eee 


=5 3 ' * A 
N GVLV GY, = By X 


nTtbetytenapne % 
hada ue Ek 

* i iP *toaA 

Y. See eae, x, 


Since the first k eigenvalues and eigenvectors of X are iden- 
A A 
tical to those of Xo the known data matrix X is substituted for X» 
giving us the solution 


X (6) 
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where all terms on the right hand side of (6) are known. 


“A 


KoO* 
The Eckart-Young procedure shows that if x = By Vy 


> oS mG ~ ae is a minimum for the selected k, i.e., sums of squares 


then 


* 
of errors are minimized, where By may be interpreted as the factor 


* 
loading matrix, YE may be interpreted as the factor score matrix. 


Summarizing, Tucker used the formulation 


A 
ose a MRS he 
Ce re 1/2 
= (N V,.G,) (N W) 
gee een 
es 


Within the context of component curve analysis there are a 
number of unresolved issues. Thus there are a number of decisions, each 
containing an element of subjectivity, that must be made by the experi- 
menter who decides to use a component curve approach. These are discussed 


in the following four sections. 


Issue on Choice of Matrix 

Ross (1964) has discussed the difference in factor structure 
resulting from the factorization of correlations, covariances or cross- 
products and recommends the use of cross products for factoring learning 
data since this method retains information on both means and variances. 

Tucker (1956) noted that the removal of trial means from the 
data matrix is appropriate when the scale of measurement is consistent 
for each variable over the population of people, that the removal of per- 
son means is appropriate when the scale of measurement is consistent for 
each person over the population of observations, and concluded that the 


use of a double centered score matrix may be appropriate when both 
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conditions are satisfied. 

The type of simulation game used in this study consists of pre- 
senting the student with a set of variables, some of which the student is 
allowed to modify directly and some of which are modified as a result of 
changes made to other variables. From a general point of view the task 
and the setting remains the same for each trial, but the actual values 
of the variables will change from trial to trial depending upon the de- 
cisions made by the student. 

Within the context of analyzing student performance data on 
such a computer-based simulation game, Ross' (1964) suggestion of using 
the cross-product matrix seems the most appropriate since this method re- 
tains all of the available information. However, to the extent that one 
can consider the trials to be exactly the same, it is desirable to also 
investigate the nature of the solution derived from the variance-covari- 
ance matrix (i.e. trial means removed); to the extent that one can consi- 
der each subject having a propensity towards one type of approach to the 
Simulation game, the raw ee ee with the person means removed should 
be analyzed; and to the extent that both conditions are assumed to be 
valid, the double centered score matrix should be used. 

The present study will investigate the results from analyzing 
the four matrices mentioned above, namely the raw data matrix, the data 


matrix with trial means removed (variance-covariance matrix), the data 


matrix with person means removed, and the double centered data matrix 


(both trial and person means removed). 


Issue on Number of Components 
Tucker (1960, 1966) used two different criteria for determining 


the number of components that should be used to adequately describe the 
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I 
data. The first of these criteria was a mean Square ratio. 
Let nN represent the data matrix for n trials and N individuals 
and let A, represent the p th eigenvalue resulting from an Eckart -Young 
decomposition of X. Then Tucker points out that the following equations 


involving sums of squares are valid. 


n N 9 min (n,N) 9 
= = * ji x: = Vp 
j=l i=l p=1l 
n N k 
a2 2 
= cata = Ap 
j=l i=l p=1 
n N min (n,N) 
2 : 2 
a Ne = Ap 
j=l i=l p=k+l 
n N n N a2 n N 2 
ere a> > aioe Mla —. ity 
j=1 i=1 j=l i=l j=l i=l 


In order to determine the importance of the k th factor, Tucker 
considers the following ratio of sums of squares where an adjustment must 


be made for degress of freedom in order to have a mean Square ratio: 


ri 


min (n,N) 


ra 


p=k+1 





Since Tucker does not provide a derivation for this formula, 
when he suggests that it is approximately distributed as F, the following 
explanation is given. The degrees of freedom for the data matrix is equal 
to the number of independent observations, nN. Let the matrix be parti- 


tioned as follows: 
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Thus the total degrees of freedom may be partitioned into four 
components, 

pN =k + k (N-k) +k (n-k) + (N-k) (n-k). 
Tucker then sets the degrees of freedom for the errors of approximation, 
(DF) » equal to (N-k) (n-k), 
and the degrees of freedom for the first k factors. (dr) ék? equal to the 
remaining degrees of freedom, 

(dE) gy =k? + K(N-Kk) + k(n-k) 
To determine the degrees of freedom for just the k th Tractor, (df) the 


following equation must be solved: 


(df), (df) dies (df) 


=(k-1) 


k> + K(N-k) + k(n-k) - [OK-1)2 + e-1)(NekHL) + (k-1) (n-k+1} 
=n+N+1 - 2k 
Therefore Tucker ends up with the following mean square ratio for the k 
th factor: 
2 2 
Ay (NK) n-k) AL SUPE 


(MSR). = 
k 2 Z 
=a, (n+N+1-2k) =" (df), 


Tucker pointed out that the distribution of this ratio is un- 
known, but that the F distribution provides an adequate approximation for 


large sample sizes. Thus at present this approach does not provide a 
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decisive decision criteria for the number of components to be retained 
but it does provide an approximate test. 

Tucker's second criteria for determining the number of compo- 
nents was based on the concept of external validity. If a learning com- 
ponent curve is to be considered meaningful (ie. it represents a consis- 
tent learning behavior instead of a fluctuating random behavior) it should 
form a smooth curve. Therefore a subjective, visual review of the plotted 
curves provides the second criteria. Tucker has provided a quantitative 

2 Aa 
measure, d , of the smoothness of a curve, where d° is equal to the sum 
of squares of the differences between consecutive coefficients of the curve. 
Unfortunately at present there is no statistical test associated with this 
measure. 

Weitzman (1963) has provided a third approach to the problem of 
the number of factors to be retained. 

Since the first factor component curve accounts for the most 
variance possible and thus approximates the over-all mean 
curve and since all the factor component curves are mutually 
orthogonal, the remaining factor component curves should fluc- 
tuate about the zero line. Any significant trends in this 
fluctuation will be indicated by the runs test, if the runs 
consist of pluses and minuses representing fluctuations of 
the factor component curves above or below the zero line. 

When all three criteria give the same result reasonable confi- 


dence should be placed on the decision. It is expected that the three 


criteria will be in close agreement for the majority of situations. 


Issue on Shape of Component Curves 


The interpretation given to the shape of the components is also 
subjective. Tucker considered data from a probability learning task 
where the raw scores consisted of the number of correct responses for 


each subject over twenty-one consecutive sets of twenty presentations. 
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He characterized a curve that was well above zero at all times as early 
learning, a curve that started near zero and continued to rise as middle 
learning, and a curve that started near zero and follwed the base line 
for about six trials before starting to rise as late learning. Weitzmann 
appears content to note the existence of more than one curve and does 
not provide an interpretation of the individual curves. It is also pos- 
sible to interpret large deviations from zero as accounting for a larger 
proportion of the variance at that particular trial, 

A review of two studies (Tucker, 1960; Burnett, 1971) indicates 
that Tucker's component curve analysis may produce component curves with 
approximately the same shape. Thus each successive component curve ap- 
pears to have a wavelength approximately one-half that of the preceeding 
component curve. This study will provide further information oa this 
topic by comparing the shape of the component curves resulting from an 
analysis of student performance data on a simulation game with the con- 
ponent curves resulting from an analysis of data that simulates "random 
performance", 

The issue on the shape of the component curves is further com- 
plicated by the problem of rotation, since the shape of the curves will 


vary with the rotation of the axes, 


Issue on Rotation of Axes 
There are two principal reasons for rotating the axes: 
(1) to enhance the meaningfulness of the learning curves, 
and 
(2) to provide maximum separation among the subjects. 


At present there are no firm guidelines as to what constitutes a meaningful 
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or desirable rotation. The relationship between rotation and the above 
considerations will be discussed separately, First, consider the concept 
of meaningfulness of the component curves. As was mentioned previously, 
the component curves should be relatively smooth - a ragged sawtooth 
curve does not fit our conception of a learning curve. Therefore any 
rotation (i.e. transformation) that noticeably reduces the smoothness of 
the component curves would be considered undesirable, Also the general 
shape of the curves should be interpretable within the context of the 
learning task, in this case a simulation game. Both of these criteria 
require a subjective evaluation of the results from a particular rotation. 
However it is at least equally important that the theoretical basis for 
a& particular rotation be relevent to the given situation. For example, 
a rotation may tend to distribute the variance over the derived component 
curves, The issue thus becomes one of deciding whether it is better to 
emphasize the maximum possible variability at each step or whether one 
should identify the different component curves and then attempt to equa- 
lize their contribution to the variance accounted for. Another feature 
of rotation deserves mention. A linear transformation, such as a rota- 
tion, applied to a set of component curves does not affect the canonical 
correlation with another set of external measures. Therefore nothing is 
either gained or lost with respect to the overall relation between the 
two sets of data when a rotation is performed. 

It follows that a rotation designed to increase the separability 
among the subjects will not have a detrimental effect with respect to the 
relation with a set of external measures, considered as a set. One natural 


approach to this objective is to apply a simple structure criterion to 
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the set of scores on the component curves (i.e. in the person space), 
This possibility will be intestigated in this study. 

In order to clarify the objective of the present study with res- 
pect to the issue of rotation, it is convenient to classify rotations 
into four groups depending upon the criteria for rotation: simple struc- 
ture in the trial space, simple structure in the person space (which will 
be investigated in this study), rotation to a specified target, and rota- 
tion to maximize or minimize a particular criterion. The restriction to 
an orthogonal rotation appears desirable since this ensures the indepen- 
dence of components. It is possible that oblique rotations will come in- 
to use after more development with the procedures has occurred. 

Tucker considered the criteria of simple structure in the trial 
space but rejected the idea for theoretical reasons - he was unable to 
justify why one would want many zeros in the component curves over trials. 
This possibility was also investigated by Burnett (1971) who applied 
quartimax, varimax, and equamax rotations to the trial space but no noti- 
ceable improvement in simple structure was noted. Thus the concept of 
simple structure in the trial space does not appear to hold much promise, 
from either a theoretical or practical point of view. 

Tucker also looked for simple structure in the person space to 
see if individuals followed different component curves but the results 
were negative. He finally used a graphical procedure for rotation such 
that all component curves had non-negative entries, non-negative slopes, 
and that they all reached an asymptote. Burnett (1971) applied quartimax, 
varimax, and equamax criteria in the person space and achieved more en- 
couraging results - approximately eighty percent of the students could 


be identified with a single component. Therefore these rotations will be 
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considered in the present study to provide further information on these 
methods of rotation, 

The methods of rotation to a specified target and rotation to 
maximize or minimize a particular criterion require a depth of under- 
standing about both the nature of performance data on a simulation game 
and the characteristics of component curve analysis that is not available 
at the present time. Thus an investigation of these methods appears pre- 
mature until further research is carried out in both simulation gaming 


and in component curve analysis, 


Il. SIMULATION TEACHING TECHNIQUES 


Classification of Simulation Games 
No universal taxonomic system exists for classifying simulation 
models. Moss (1958) has suggested a five point scale based on the degree 
of abstraction. Naylor (p. 16) uses an arbitrary four category scheme, 
Samuelson (p. 315) has proposed a sixfold classification of dynamic sys- 
tems, The type of model to be considered in this study falls into his 
sixth group, dynamic stochastic and historical, 
Beck and Monroe (1969) emphasize six dimensions that may be 
used to describe simulation models: 
l. Reality - the degree of fidelity to a real life situation 
2. Complexity - includes consequences for the learner, response 
choices, social factors, and constraints on time 
or length 


3. Curriculum 
Content - determined by the learning objectives, 
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characteristics of the learner, and bias of the 
designer 
4, Model design - a synthesis of the mode in which the mode 1 
Operates (eg: computer-based) and the degree of 
mathematics used 
5. Replicability - allows for identical trials 
6, Evaluation - related to teaching-testing cycles 
Any description of a simulation exercise should include sufficient infor- 
mation to allow a clear judgement of the model's characteristics as they 


relate to these six dimensions, 


The Issue of Realism Versus Abstraction 

Verba (1964) defines simulation as a dynamic model of a system. 
"Other models...may attempt to represent a system through verbal means, 
mathematical means, or pictorial means, But the simulation model differs 
in that it is an operating model", The concept of an operating model is 
recognized by Raser (p. 10), Dawson (p. 3), and Naylor (p. 3). 

Boguslaw (1965) distinguishes between replica and symbolic 
models: replica models are. material and look like the real thing whereas 
symbolic models use ideas, concepts, and abstract symbols to represent 
objects. Raser (p. 11) points out that a simulation abstracts, simpli- 
fies, and aggregates, in order to introduce into the model more clarity 
than exists in the referent system. Which components and relationships 
are included depends upon the purpose and goal of the experience, 

Thus many training programs use simulations that involve a high 


degree of realism. However, other simulated situations may use a highly 
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simplified model in order to focus attention on specific variables. This 
is elaborated upon in the following three sections which discuss the 


range of simulation usage within an instructional frame of reference. 


Training Programs 


Perhaps the most familiar application of simulation is in the 
area of training. The importance of this technique was brought into the 
public spotlight recently as information was released on the training pro- 
gram for man's first journey to the surface of the moon. Simulation is 
also widely used for training airplane pilots and for automobile driving 
instruction. In all of these instances the simulator is a relatively ac- 
curate reproduction of a complex man-machine-environmental system. 

Jacobson (1966) mentions the need for a simulation that pro- 
vides internal and external visual cues of a high degree of realism for 
the training of astronauts in the Gemini program. Chapman's (1959) des- 
cription of the RAND Air Defense Simulation experiments and Cohen's (1960) 
description of the Carnegie Tech Management game both emphasize the ex- 
ternal realism of the simulation situation. An excellent discussion of 
the objectives and results of using the Carnegie game is presented by Dill 
and Doppelt (1965). 

Tansey (1970) reviewed a number of simulation techniques used 
in the training of teachers. Developments at the Oregon State System of 
Higher Education, Indiana University, Michigan State University, Univer- 
sity of Tennessee and the Berkshire College of Education in England were 
all discussed. The general theme of all the projects centered on the 
need to integrate the theory and practice of education more closely in 


the training of student teachers. The projects did not rely on a high 
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degree of realism with a classroom situation - the Oregon system presented 
a number of classroom incidents via a film projector and the student was 
expected to respond in an appropriate manner, in Berkshire the initial 
situation was presented via a printed report and the student teacher was 


required to give a written reply describing his action. 


Use of Simulation Games as an Instructional Technique 


Practical and economic difficulties impose restrictions on the 
types of simulation that are suitable for use in a school system, At 
present the majority of such techniques consist either of situations that 
primarily involve the interaction of students, such as social simulation 
(SIMSOC, GHETTO, COMMUNITY DISASTER, and INTER-NATION) or situations that 
require very simple, if any, computations (CARIBOU HUNTING, GAME OF RAIL- 
ROADING, WATERLOO, and MARKET GAME). 

Zuckerman and Horn (1970) have prepared a bibliography of 404 
games which includes brief descriptions for the games mentioned above. 

The authors estimate that, "the 404 games listed represent only a portion, 
perhaps as little as a third, of the games presently in use (p. 1)." 

Because of the "game" feature, many of these exercises are be- 
ing developed by private companies for sale to the general public. The 
resulting proliferation of such games has definitely outstripped accom- 
panying research. Boocock and Coleman (1966) and Boocock (1966) have dis- 
cussed the effectiveness of three non-computer based simulation games 
from both a motivation and learning point of view. Cherryholmes (1966) 
suggested that the case for learning and attitude change resulting from 


simulation games may not be as strong as has been claimed, but this is 
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qualified by the statement, "One serious weakness in much evaluatory re- 
search has to do with the criterion problem - i.e., specifying what a 
given set of materials is supposed to teach and then devising tests to 
measure this accurately". This receives further elaboration in a later 
section of this study, Evaluation of Simulation Games, 

The High School Geography Project (Watson, 1969; Helburn, 1967; 
1968; Kurfman 1967) represents one attempt to integrate simulation gaming 
directly into a course, rather than to just use a simulation game as "en- 
richment" material. It seems fair to conclude that the use of simulation 
games is experiencing a popular rise in prominence - but that research 
into its effectiveness has lagged behind. The emergence of computer tech- 
nology should provide further impetus to both aspects, as discussed in the 


following two sections. 


Advent of Computer Based Simulation Games 


With the advent of computer technology a new level of simula- 
tion exercise has become possible. Models that involve complex calcula- 
tions, models that contain complex interaction among variables, and models 
that provide individualized feedback to the student are now possible. 
These models would be classified by Boguslaw (1965) as symbolic, however 
the form of the output on the printer, cathode ray tube, and slide pro- 
jector are all intended to add realism to the exercise. A new develop- 
ment which may have far reaching possibilities is the generation of holo- 
grams (three dimensional pictures). Hendren (1968) is interested in the 
computer generation of synthetic holograms in order that designs may be 
viewed in three dimensions, from different viewpoints, and in motion, 


Wing (1967) has developed three economic games for sixth grade 
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students: the Sumerian Game, the Sierra Leone Game, and the Free Enter- 
prise Game. The sophistication of these games is only made possible by 
the use of a computer. Lagowski (1968) has helped develop a chemistry 
course that simulates the laboratory experiments. Naylor (p. 10) mentions 
a number of sophisticated simulation models: the shoe, leather, and hide 
industry, the West Coast lumber industry, the United States economy in 
recession, and a behavioral theory of the firm. However at present these 
latter models are being used to gain a further understanding of the pro- 
cesses involved and are not yet designed for use as an instructional 
technique, 

Lekan (1969, 1970) has compiled two bibliographies for computer 
assisted instruction materials. In 1969 there were 36 programs that were 
classified as using simulation, whereas in 1970 the number had inerenced 
to 84, These programs are used in biology, chemistry, computer program- 
ming, economics, education, medicine, political science, management and 
mathematics courses, 

Although a number of programs are presently being developed, 
the report by Wing (1967) represents one of the few published studies 
that has evaluated the use of simulation models as an instructional tech- 
nique. The main purpose of his study was to evaluate the effectiveness 
of simulation in teaching economic understanding. Wing enumerates four- 
teen conclusions arising from the study, with the general trend support- 


ing the use of simulation. 


Evaluation of Simulation Games 
The objective evaluation of simulation games and the inter- 


related issue of evaluation of subjects' performance on a simulation 
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exercise is still at an embryonic stage. Tansey (1970) describes the 


present situation as follows: 


What is now urgently required is research that is not con- 
cerned primarily with the production of materials but with 
finding evaluative systems that will enable the worth of 
the basic technique to be measured. ...The setting up of 
simulation exercises is a laborious and often expensive 
process, and we urgently need accurate information in 
order that we can feel a measure of certainty that it is 
worthwhile. 

Attempts at evaluation may be categorized into two main appro- 
aches: those that rely on some form of post-simulation test and those 
that obtain a measure directly from the subject's performance on the si- 
mulation exercise. 

Examples of the post-simulation test approach include those of 
Burke and Sage (1970), and Clarke (1970) who used an instrument based on 
Osgood's Semantic Differential to obtain scales that measured concepts 
related to the simulation exercise. Burke and Sage observed the hypothe- 
sized direction of attitude change as a result of the simulation exercise, 
for 9 of the 10 concept scales. Clarke concluded that students who parti- 
cipated in the simulation exercise displayed a consistently more positive 
response to the relevant concepts than did a control group. Stahl (1970) 
used a similar procedure to examine the degree to which an individual is 
effectively involved in a number of simulation problems presented via 
different media. Using the moods adjective checklist (MACL) he obtained 
scores on factors such as anxiety, elation, fatigue, and nonchalance, 
both before and after the simulation exercise. A total of twelve mood 
factors were considered. Eight factors displayed a significant change 


after a video tape presentation, two factors had a significant change 


after an audio presentation, and three such factors were noted after a 


ac 




















z Singeidne aa Seb ive- ae 


4/3? vociaoreh (0 Ob? vosmee eae 
‘evollod @2 Rotigudte Jaeapag’ 
aya ron el.tett davee) ey shomertgaess 7! fonpys wou SP 2a ms 
isiv: Jd B42 e7p QS he oeoga sy sku “ll ipetiag bern 5 
} j he | rans 1 iw Jari} snes rye satin ivf ued tw _—_ 
14 it. /siiganes 8! ct ouninies! siege 283 .. 
nits re UOtzverizi-en at *42ro7arve nortpeensse _ 
S$ pTUNSEEOGAR Vil esaty Sw Ok .eewOoTYG 
4 Sivanenm 4°! wet om Ieht webvo 
. £l teatro 
} ig st} TAD Hf v {tO7 2 [ei a) LA alvqmed JA 
ori Ia i“ fp ler Jant3 ee ord 
' — 
4 (goat 48 teen a olsjdontl 
eatoyexs Woks 
fs ‘6. 3 <F esiqmaxd 
; : : a 
ter dam)” tu eu Ore LGVOL) sien (9 « (OV8i) aged tas 
7" *e 8, 
. 1), aelaons 320 shinemed a*Gen 
ane 
| io Age? -brawady 6 owte ot) of begete 
; » oe 
[2° Yo Fivasi’ a ce Dacian 1/230 Io nolioessee 
® 7 
I 16 4 ry | beahufon Ss Sa06 Ty) .2et eos ‘i, @wo0Ds ld on? lo J i9 
=) ¥| iat f t Eaves Ih oat he r ‘ i in tte dw ssis at des 
: “< 
4 - + 
OVOL) idase .. query Loinee a BE us a 220 Idevels? @f9 oF os 


#2 faubsyihitivns dotdy 03 sé tudly af 


piv badasessg 
a6 a 


> 


msidorg aotded wuts 













16 


@ 
a 
LP a L 7 
4 ~ 
4 4 Pf * . 







, : Oe 

anime» of seivbaségqg tal inde & 

’ F a \ aii 

Issn 8 mt Devloval yiewise 
: 7 ' 


> 


B ation ad? ante istbow sane 
Koy S55 a = \ 







aa 


4 


ve 


written presentation. The traditional post-test method for measuring the 
learning of facts and principles was used in the six studies reviewed by 
Cherryholmes (1966), and was also used in studies reported by Boocock 
(1966), Wing (1967), and Anderson (1970). Anderson concluded, "... 
learning factual information about installment credit through a simula- 
tion learning game is as effective as reading, discussing, and reporting 
about the use of installment credit in a conventional classroom approach 
---". In general, few significant differences have been observed between 
experimental groups (simulation gaming) and control groups (classroom 
approach). 

Kersh (1963) has provided one study where the subject's score 
was a direct function of his performance in the simulation Situation. He 
applied rating standards to the alternatives for each classroom problem 
in a teacher training situation and summed the scores over a sequence of 
twenty problems to obtain a composite rating for each trainee. 

Final results on any of the economic games would represent an- 
other simple measure of student performance. However other methods which 
take explicit account of most, if not all, of the available information 
on a student's performance would seem to merit further consideration. 


Component curve analysis represents a step in this direction. 


IIIl. EXTERNAL MEASURES OF SUBJECTS' CHARACTERISTICS 


The French Kit of Cognitive Factors 


Both Duncanson (1966) and Lemke, Klausmeier, and Harris (1967) 


have shown relationships between learning tasks and the factors measured 
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by the Kit of Reference Tests for Cognitive Factors collected by French, 

Ekstrom, and Price (1967). Reed (1966) explored the relationship between 

a measure of conceptual complexity, the Paragraph Completion Test and the 

cognitive factors measured using the French kit and concluded that, "the 

general reasoning factor involves an ability to structure problems and 

to deal with complexity, where complexity is defined by the number of 

elements and relations in a problem". Wampler (1966) has used several 

of the tests from the French kit to successfully predict performance in 

college mathematics. 

The Kit of Reference Tests for Cognitive Factors provides a set 
of marker tests for twenty-four cognitive factors. Five of these factors 
appeared to have particular relevance for the present study. These fac- 
tors are described by French et al as follows: 

Induction: associated abilities involved in the finding of general con- 
cepts that will fit sets of data, the forming and trying out 
of hypotheses. 

Marker tests: 1-1, Letter Sets Test (grades 8-16)(14 minutes) 
1-3, Figure Classification (grades 8-16) 
(16 minutes) 

Associative (Rote) Memory: the ability to remember bits of unrelated ma- 
terial. Tests requiring recall of items in isolation do not 
have a loading on this factor. It is possible, although there 
has been no clear demonstration yet, that this factor repre- 
sents the ability to form and remember new associations quickly. 
Marker tests: Ma-2, Object-Number (grades 6-16) (10 minutes) 

Ma-3, First and Last Names Test (grades 6-16) 


(10 minutes) 
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Number Facility: the ability to manipulate numbers in arithmetical op- 
erations rapidly. There is a little evidence that certain 
kinds of manipulation of symbols other than numbers can load 
tnis factor, 

Marker tests: N-1, Addition (grades 6-16) (4 minutes) 
N-3, Subtraction and Multiplication (grades 
6-16) (4 minutes) 
General Reasoning: The ability to solve a broad range of reasoning pro- 
blems including those of a mathematical nature. 
Marker test: R-1, Mathematics Aptitude (grades 6-12) 
(10 minutes) 
R-4, Necessary Arithmetic Operations (grades 
6-16) (10 minutes) 

Verbal Comprehension: the ability to understand the English Language. 

Marker test: V-1, Vocabulary (grades 7-12) (8 minutes) 
V-2, Vocabulary (grades 7-12) (8 minutes) 
A copy of the test direction page, which includes examples, for 

each of the above tests may be found in Appendix A. 

A review of the descriptions provided by French indicated that 

a positive relationship should exist between student performance on a 

simulation game and the factors of induction, associative (rote) memory, 

number facility, and general reasoning. It was possible to postulate 
that students may not attach the same meaning to either the instructions 
or the variables used in the model, therefore marker tests were included 


to measure the factor of verbal comprehension. 
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Matching Familiar Figures Test 


Another relevant dimension for the present study was that of 
impulsivity. The temporal stability and inter-task generality of a ten- 
dency toward fast or slow decision times to problems with high response 
uncertainty has been demonstrated (Kagan, 1966b). This dimension was 
measured by means of Kagan's Matching Familiar Figures (MFF) test, modi- 
fied for a computer-based presentation (see Apprendix C). 

In this test the subject is shown a picture of a familiar ob- 
ject, called the standard, and six similar stimuli, only one of which is 
identical to the standard (Figure 2). The subject is asked to select the 
one stimulus that is identical to the standard. Errors and response times 
are recorded for each of the twelve test items. 

Kagan has used two different measures of impulsivity. In one 
Study (1966b), "Impulsive children were above the median on total errors 
and below the median on average response latency (for all 12 items) for 
their sex. Reflective children were below the median on errors and above 
the median on response latency." However in another study (1966a) Kagan 
used both the above method and another procedure which only considered 
the response latency time. He concluded, "We have never found qualita- 
tively different results when we used response time alone or combined 
response time with errors as the index of the reflection-impulsivity di- 
mension.'' Thus the response latency time could prove useful for provid- 
ing a single direct measure of impulsivity during an actual simulation 
exercise, since in this situation a subject's response usually cannot be 


scored as correct or incorrect. 
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CHAPTER IIIL 


PROCEDURE 


I. DESCRIPTION OF SAMPLE 


The subjects for this study were all sampled from the popula- 
tion of Grade Eight students at Wellington Junior High School in Edmonton, 
Alberta. The students at Wellington Junior High School are drawn from 
four widely differing socio-economic districts and represent a general 
cross-section of the low to high-middle income groups, and in this sense 
may be considered representative of the majority of students in an urban 
environment. 

There were two principle reasons for selecting grade eight stu- 
dents. First, the restriction of sampling from one grade level minimizes 
age differences and thereby provides a form of control over this source of 
variation. This helps to minimize the confounding effect of age on the 
cognitive factors and also helps to minimize the effect of an age factor 
related to maturity and sophistication which could have some relation to 
operating an abstract exercise such as a simulation game. 

Second, since the simulation game used in this study models the 
salmon fishing industry of British Columbia, it was desirable to have stu- 
dents whose normal curriculum would include an examination of the fishing 
industry. This is currently considered a part of the grade seven Social 
Studies curriculum in Alberta. Grade eight students were selected instead 
of grade seven students to avoid the possibility that some students may 


have studied fishing just prior to participating in the study. Thus grade 
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eight students satisfy the following criteria: 

1. Age is similar to that of students who normally discuss the 
fishing industry in their curriculun. 

2. Prior knowledge of the fishing industry is approximately 
equal - since almost a year had elapsed since they had last 
studied the fishing industry, 

The actual students used in the study were sampled according to 
the following precedure. First, a complete list of all grade eight stu- 
dents registered in Wellington Junior High School was obtained. The com- 
plete list totalled 274 students from 9 different classrooms. Each stu- 
dent was then assigned a unique number from the range 1 to 274, The 
APL 360 System (Falkoff and Iverson, 1968), using the function 75 ? 274, 
‘ie used to obtain a vector of 75 random numbers, without replacement, 
from the range 1 to 274. The 75 students whose numbers matched those that 
were obtained from the random number generator were used in the study. 

Each of the selected students was given a letter asking for 
their parents' permission to participate in the study. A copy of the form 
letter is included in Appendix E. All of the parents gave their consent 
to having their children take part in the study. 

Although 75 subjects were selected by the random procedure, only 
67 subjects were included in the analysis. Eight students were excluded 
from the final analysis for the following reasons: 1 student moved, 2 
students had incomplete data for the simulation game, 1 student had in- 
complete data from the set of marker tests for cognitive factors, and 4 
students were absent when the marker tests were given. A further expla- 


nation regarding the issue of incomplete data is necessary. The incomplete 
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data for the simulation game for two students was caused by an error in 

the computer program for the simulation game. This error was subsequently 
corrected and no further problems were encountered among the other students. 
Incomplete data on the marker tests for one subject was caused by a mis- 
understanding on the directions for one of the tests - as a result the sub- 
ject failed to complete one of the tests. 

The 67 students composing the sample for this study consisted 
of 28 boys and 39 girls. The random sampling procedure selected students 
from all 9 classrooms. Their age varied from 12 years, 1 month to 14 
years, 10 months - representing a range of 2 years, 9 months. The mean 
age was 13 years, 6 months and the standard deviation was 6.26 months. 

The father's occupation for each of the students in the study 
was obtained from the school records. Forty-four of the sixty-seven re- 
cords were sufficiently precise to permit the assignment of a value from 
Blishen's (1967) Socio-Economic Index. Occupational information for an 
additional fifteen subjects was obtained from Henderson's Edmonton, 
Alberta City Directory (1970). Thus a total of 59 of the 67 subjects 
(88%) were used to calculate a Socio-Economic Index for the Wellington 
sample. This scale is based on the 1961 Canadian census data and takes 
into account information on income, education, and prestige - as measured 
by the Pines - Porter occupational scale (Blishen, 1967). The resulting 
Socio-Economic Index covers 320 occupations, ranging from Chemical Engi- 
neer (76.69), Dentist (76.44), and Professor (76.01) to Shoemaker (26.56), 
Fish Canners (26.09), and Trappers (25.36). 

The data from the Wellington Junior High School District is com- 


pared with that of Alberta and Canada in Table I. The Wellington area 
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TABLE I 


COMPARISON OF WELLINGTON JUNIOR HIGH SCHOOL DISTRICT 
WITH ALBERTA AND CANADA 
USING BLISHEN'S SOCIO-ECONOMIC INDEX 








SOCIO-ECONOMIC PERCENTAGE OF LABOUR FORCE 
INDEX WELLINGTON ALBERTA CANADA 

70+ 3 
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30 - 39 44 29 32 
Below 30 |e) S13) 3L 





Note: Alberta and Canada figures are based on the 


1961 Census. 
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had a mean Socio-Economic Index value of 39.15, with a standard deviation 
of 11.68. This compares with the Alberta figure of 39.20, standard de- 
viation 12.64 and a Canadian value of 38.81, standard deviation 12.19. 
The results are supportive of the statement that the socio-economic 
background of the Wellington area is similar to those of both Alberta 


and Canada. 
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Il. DESCRIPTION OF THE IBM 1500 INSTRUCTIONAL SYSTEM 


The IBM 1500 Instructional System is a computer controlled sy- 
stem of remote terminals and peripheral devices that has been designed to 
provide computer-assisted instruction for students. The main computer is 
an IBM 1130 with 65,000 characters of core memory supplemented with about 
five million characters of on-line disc storage. There are two magnetic 
tape units which can provide virtually unlimited storage capacity. 

The system is capable of handling thirty-two student terminals 
which interact with the IBM 1130 on a time-sharing basis. The present 
System operated by the Division of Educational Research Services at the 
University of Alberta has eighteen terminals. Sixteen terminals consist 
of a cathode-ray display screen (CRT), a light-pen, a keyboard, an image 
Projector, and an audio unit. The two remaining terminals are remote 
typewriter devices. Fifteen of the complete student terminals plus one 
of the remote typewriters are located in one room, where students take 
the various courses chee. aes available. 

Since the System is based on time-sharing, student performance 
at each terminal is independent of the other terminals. Thus each student 
may be taking a different course or all of the students maybe at different 
positions in the same course. A student at a terminal gains the impres- 
sion that the resources of the computer are being devoted specifically to 
his requirements since appropriate messages are displayed within a couple 


of seconds of his individual responses. 
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I11. DESCRIPTION OF SIMULATION GAME 


| The computer program, called FISHY, that was used in this study 
Simulates the salmon fishing industry of British Columbia. The program 
was written in a language called COURSEWRITER II for use on the IBM 1500 
Instructional System. A copy of the program documentation for FISHY may 
be found in Appendix B. 

The initial frames (CRT lay-outs) provide a general description 
on the importance of the fishing industry in Canada. There are 8 such 
frames preceeding the actual start of the simulation game. Thus the ini- 
tial introduction to the game consists of the following sequence of frames: 


frame 1: The fishing industry has always played an important role in 
the economic development of Canada. Although the fishing in- 
dustry has declined in relative importance in the overall 
economy, Canada remains one of the world's leading fishing 
nations. 


frame 2: This game is about the salmon fishing industry in British 
Columbia. Salmon is, by far, the most important fish found 
in British Columbia waters. Because salmon is high in food 
value and is easily canned, British Columbia salmon is sold 
all over the world. 


frame 3: There will be some lighted squares (like this, ZY) on the 
screen. These squares, or 'targets', will always follow a 
short description of the type of decision they stand for. 


frame 4: To make a decision: 
1. Point the light pen at the target, Q ,» that indicates 
the type of variable that you want to change. 


2. A question will then appear at the bottom of the screen. 
You will then type in a number that gives the amount of 
change you want to make. 

That is, a decision requires 2 steps: 

Step 1. Indicate what you want changed. 


Step 2. Type how much the change will be. 
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frame 5: After you have made your decisions for a "year', point the 
light-pen to the target after the word 'END'. This indi- 
cates the end of the 'year'. The results of your decisions 
will then appear on the screen and you will be ready to start 
making decisions for the new 'year'. To continue, point the 
light-pen at the target after the word 'END'. 


END Z 


frame 6: Please remember that this exercise will have many of the 
changes that would actually occur in a major industry. Thus 
one would expect that there will be a number of changes that 
will occur in addition to the decisions that you make. 


frame 7: For example, equipment will depreciate in value, men will 
leave their jobs, raises in salaries may occur, the health 
of the salmon population will depend on the quality of its 
environment (which in turn is affected by the importance of 
fish ladders and pollution control measures), and the market 
for fish will vary depending on world-wide conditions. Thus 
in general it may be said that the industry will always be 
in a state of change. 

frame 8: Note: The numbers used in the game are much smaller than in 
the real fishing industry. This should help to make the game 
easier. Since the value of your decisions will affect the 
amount of money you will make, your 'score' will be the total 
cash on hand. The game will last for a period of 10 years, 
from 1960 until 1970. Type in the year '1960' to start the 
game. GOOD LUCK and GOOD FISHING! 

The next frame is pictured in Figure 1, which represents the 
actual CRT-layout used for the entire game. 

Thus the student could change the number of workers (by hiring 
or laying-off), or add to the value of equipment, fish ladders, or pollu- 
tion control. Decisions involving monetary values were immediately re- 
corded by an appropriate increase in the indicated variable and a corres- 
ponding decrease in the cash on hand. When the student pointed his light 
pen at the target following the word END, the following message was dis- 
played at the bottom of the screen, "The calculations for the current year 


are now being performed. Notice that the values displayed above will now 


change. Press space bar to see changes". The actual equations used in 
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the simulation model are included in the program documentation for FISHY, 


found in Appendix B. 
IV. DESCRIPTION OF COMPUTER-BASED MFF TEST 


This program, called FIGUR, provides a computer-based presenta- 
tion of Jerome Kagan's (1966a) Matching Familiar Figures (MFF) test. This 
test may be used to provide measures on an "Impulsivity - Reflectivity" 
scale. This version of the test requires a student to select a matching 
figure from among six choices. For each trial the student is shown a 
picture of one figure separated from a set of six figures by a solid 
line. The student is required to select the figure from the set of six 
that is identical to the single figure. 

The test contains two practise trials: 


vial es cups 
fies ye “Ae rulers 


Following are twelve trials, presented in the order: 


Tera s.3..: houses 
AM igs MCP scissors 
Leta ro ee: phones 
Ces Late oe bears 
(Ges ho Ry ee trees 
i jess tw Bee | leaves 


jhe wh tee BR cats 
iMeiewe Nap dresses 
{gab yh Wl animals 


erate ees lamps 
Reale oe. boats 
Trial 14 : cowboys 


An example of one of the trial pictures is provided in Figure 2. 
If a student selects an incorrect choice during the two practise 


trials a message is displayed giving the reason why he was wrong and he 

















\ 1. =p ine 
ii wy Sebe + oth Lohow woktelgeks aly > 


2 »pluiaqigh aa ono 


i 


roal. Fae feet ae * 10D 46 9OTTSTeem, GAS - 


shiverzg RABE bol apggona ia tdT og 


; : Bs q I8% Oh Mbit TAM Uae] >" rgen ano st. Yo ¥ 


ip NO te Wen ebivetg, Oa eam Se eee ee 
ngohie Bw wer Lupe’ se ed? 26, haber shit us 
o 
2 ee >. .asrxlod+ gio geil, aoee: 


ae E 73 is & Molt hed ¥1o2 ‘9st o<t7 ‘26 Ssruy3. 


1 4 »; 
el 


i odJ inpal%a oF Hetiups) ae see 


cos 6 lk 
4 


f 


int be aA ke. sdt 8 lestjoshe #i 
f — 


“4 3¢73apaq ons ser Paring ago api 


Fy at A [pie : 
Rigi he ( £. Dare 


- 
i 


¢lpitd oviows 92a gee 


asAged 7 < lanyyt ~ 
e1ussiz.. > ean 


anrteriy P4 
4 


r a eoneerily 


: eC Ueat 


< | equet 


FIGURE 2 


SAMPLE ITEM FROM MATCHING FAMILIAR FIGURES TEST 
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is branched back to the same problem and asked to try again. If he is 
correct he receives confirmation and is branched to the next problem in 
the sequence. The same pattern is repeated for the next twelve trials 
except if he is incorrect he is merely told to try again - no message is 
given explaining why he was wrong. 

A copy of the program documentation for FIGUR may be found in 


Appendix C. 


V. STUDENT PERFORMANCE ON COMPUTER TERMINALS 


Each subject was assigned to one of six groups, each group con- 
taining either 12 or 13 subjects. The grouping was necessary for schedu- 
ling reasons on the IBM 1500 Instruction System - which has only 16 stu- 
dent terminals. The groups were formed on the basis of the order of 
occurrence of the random numbers used in the selection procedure. Thus 
the first 12 random numbers were used to form group 1, the next 12 random 
numbers were used to form group 2, and so on. 

In order to decrease the initial unfamiliarity with the system 
and to help familiarize the students with the method of "inputting" res- 
ponses, each of the students selected for the study was given a brief 
description entitled, "Student Use of the Computer Terminals". A copy of 
this description may be found in Appendix D. The student's ability to 
use the terminal was tested during the initial frames of the course which 
required the student to respond by using the light-pen and then the key- 
board. If the student failed to respond within a period of 100 seconds 


for the light-pen or 120 seconds for the keyboard, a message was printed 
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out at the typewriter terminal indicating the student that was having 
difficulty. A summary of the frequency of occurrence of these messages 
for human intervention is given in Table II, Approximately half of the 
students required no further assistance using the terminals. Those stu- 
dents experiencing difficulty were given additional help so all students 
were familiar with the use of the terminals before the actual simulation 
game began. 

After each student had demonstrated that he could use the ter- 
minal the actual simulation game began. Some of the students were con- 
fused at the start of the game about what was required. However the na- 
ture of the decision making was reviewed with these students and after 
the pattern of one or two decisions was established no further difficul- 
ties were encountered. The results of the students decisions Worede tae 
ated when the student indicated 'end of year' and the process was continued 
for ten cycles or 'years'. After 10 years, a summary of the student's 
cash-on-hand at the end of each year was displayed on the screen. 

The computer program then branched to a new course segment which 
consisted of a computer-based version of Kagan's (1966a) Matching Familiar 
Figures test. There was an initial problem with some of the terminal 
image projectors - the pictures were out of focus. However this was cor- 
rected during the two pre-test trials and should not have appreciably af- 
fected the later results, although a difference in the sharpness of the 
images was observed among some of the terminals. 

Since the students proceeded through both the simulation game 
and the MFF test at individual speeds, the students completed this por- 


tion of the study at different times. Tables III and IV give the elapsed 
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TABLE IL 


FREQUENCY OF INITIAL DIFFICULTIES 
USING THE COMPUTER TERMINALS 


LIGHT -PEN KEYBOARD BOTH 
RESPONSE ONLY RESPONSE ONLY 


3 “ 2 
0 1 2 
2 2 0 
2 1 2 
1 1 0 
1 3 4 
9 LZ 10 
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TABLE III 


SUMMARY OF ELAPSED TIMES FOR STUDENTS 
ON THE SIMULATION GAME, 'FISHY' 











MINUTES 
Group N Lowest Highest Range Mean Sa. Da 
1 12 34.12 65.06 30.94 51.28 8.60 
2 8 29562 57.24 27,62 48°68 8.46 
3 al 21202 65.49 38.47 46.34 12.42 
4 Vi 29332 61.07 35.55 45,20 2.08 
5 L3) 27976 54,29 23,55 38,972 8.84 
6 12 42260 60.95 16.85 47.45 5.60 
Totals: 67 ve eye 65.49 39.97 45.45 9.86 
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TABLE IV 


SUMMARY OF ELAPSED TIMES FOR 
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STUDENTS ON THE MFF TEST 
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MINUTES 
Highest 


17.44 
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14.36 
EScr3 
16579 


18.13 


Mean 


10.38 
10.01 
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9.02 
10.82 
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times for the two computer programs illustrating the individual differences 
among students using computer-assisted instruction. In order to minimize 
classroom disruption and to avoid putting a "hurry-up" pressure on the 
slower students, the students were given a demonstration course when they 
finished the MFF test. Thus all students remained on the terminals until 


the last student had completed the MFF test. 


VI. ADMINISTRATION OF MARKER TESTS FOR COGNITIVE FACTORS 


The tests selected from the Kit of Reference Tests for Cognitive 
Factors (French, Ekstrom, and Price, 1967) were given at Wellington Junior 
High School the week following the students' use of the computer terminals. 
The students were divided into two groups on the basis of their home class- 
room in order to minimize the disruption of the normal school schedule. 
Thus students from classes 8A to 8E were tested during the morning and 
students from 8F to 8J received their tests during the afternoon of the 
same day. It took approximately 2 hours and 20 minutes to administer the 
10 tests to each group. 
The following pre-test instructions were read to each group: 
1. This is the second part of the study. 
2. It is not expected that you will finish all of the tests - the 
Same tests are used with adults - but please do as well as you can. 
3. All results will be known only to myself and all results will be 
kept completely anonymous. 
4. There will be a set of 10 tests. 
5. Each test will have a page of instructions followed by two or more 


actual test pages. 
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6. All tests will have time limits - clearly indicated beforehand. 
7. Open your envelopes. Be sure to keep the tests in order. 
The testing then proceeded with the tests given in the following order: 
1. Vocabulary Test (V-2) 
2. letter Sets Test (1-1) 
3. Subtraction and Multiplication Test (N-3) 
4. First and Last Names Test (Ma-3) 
5. Mathematics Aptitude Test (R-1) 
6. Figure Classification (1-3) 
7. Addition Test (N-1) 
8. Object-Number Test (Ma-2) 
9. Necessary Arithmetic Operations Test (R-4) 
10. Vocabulary Test (V-1). 
Thus any fatigue effect should distribute itself over the different 
factors. 

The students read the instructions for one test while the re- 
sults from the previous test were being collected. The students were 
asked if they had any questions on the nature of the test - if there were 
none the first timed section was begun. The only confusion regarding the 
instructions was with test, Figure Cr eeieieee (1-3), which required 
further explanation for both groups. 

Upon completion of the formal testing, the students were asked 
to indicate on a 7-point scale, where a 1 represented DISLIKE and a 7 
represented LIKE, their position with regard to the following questions: 

1. Indicate on the scale your feelings about taking part of your 


school curriculum on a computer terminal. 
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2. Indicate on the scale your feelings about the game FISHY. 


3. Did you find the testing session today too long? Answer yes or no. 


4. Did you find the testing session today too hard? Answer ves or no. 


The next chapter will give the statistical analysis and results of the 
data that was collected according to the procedures outlined in this 


chapter. 
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CHAPTER LV 


STATISTICAL ANALYSIS AND INTERPRETATION OF RESULTS 


I. COMPONENT CURVE ANALYSIS OF STUDENT PERFORMANCE 


The raw data for each subject used in the component curve 
analysis consisted of the values of the "cash on hand" variable from the 
simulation game FISHY over the 10 "years", or trials, of operation. Since 
the identification of individual differences in performance is an impor- 
tant feature of this study, the raw data are given in Table V in order 


to facilitate later comparisons with some of the derived measures. 


Mean Cross Product Matrix - Unrotated Solution 

The scale for the "cash on hand" variable was modified prior 
to the actual analysis by multiplying all values by 0.10. This reduces 
the magnitude of the reported values without altering the nature of the 
results. The method of component curve analysis was applied to the mean 
cross product matrix formed from this data. The mean Square ratios, re- 
ported in Table VI, were used to determine the number of characteristic 
vectors required to reproduce the original data. Assuming that these 
mean square ratios are approximately distributed as the F distribution, 
a decision was made to retain 4 characteristic vectors. Each successive 
curve shows an increasing amount of fluctuation, however the first four 
characteristic vectors are sufficiently "smooth" to permit a learning 
curve interpretation. These curves are illustrated in Figures 3 - 6. 


The next step in the analysis computed the loadings of the 
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TABLE V 


STUDENT DATA FROM SIMULATION GAME 'FISHY! 





(X MATRIX) 
Subject Value of 'Cash on Hand' Variable after Each 'Year' (Trial) 
mm. t 2 3 i 5 6 Z 8 9 10 Average 





1 199 188 Sm 104 a7 114 99 110 106 98 iat 
Fa ES dy Bay a BS 105 ah 134 174 209 192 145 194 
Poe her oe eee 45) | 45906lARG AGO ” 356 471 
4 | SaaS HO Ws Be 110 138 P53 LSS 163 206 218 129 159 
eo tot. OAL | OL ieee 563 6349 333 320 300 388 


Gee sU0 sao. 354 290 99524 343° 350 341 306 309 357 
ome oe Ae? 36280 352 4250) | 301 396. 358598260 Li? Jah 
a 303 243 137 65 106 64 80 100 94 oo 124 
Me Lee eee ere lea eon 355° 337%, 31L 294 272 383 
6) 64 92 100 £53 188 182 i>) 196 212- 200 154 


no ‘\ q r i\ 


acc. eS 


oD = ~9 


al 
O. od 





Subject 
ID 


365 


Value of 

2 3 
345 267 
195 211 
458 440 
203 206 
204 160 
357 422 
331 379 
412 430 
486 486 
414 454 
418 462 
39 9 
502 488 
pad § 526 
399 401 
Pe | 194 
9 s 
24 33 
456 462 
349 165 
262 281 
119 109 
290 ei 
368 352 
248 231 
462 416 
91 8 
547 256 
429 SOF 
L444 446 
314 287 
507 508 
Sa 299 


4 


282 


'Cash on Hand! 


5 


289 
190 
362 

89 
118 


385 
451 


Fa 


Variable after Each 


6 


268 


TABLE V (continued) 


7 


267 


8 


263 


9 


249 


'Year! 


10 


239 


(Trial) 
Average 


243 
163 
400 
118 
127 


312 
430 
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TABLE VI 


COMPONENT CURVE ANALYSIS OF MEAN CROSS PRODUCT MATRIX 
MEAN SQUARE RATIOS 


——————— 





Component Eigenvalue dfl (aber Mean Square Ratio 


1 10692,39 76 594 111.28 
2 554.08 74 520 19.77 
3 92.82 72 448 5.55 
4 50.92 70 378 aly 
2 22601 68 310 3.42 
6 13.18 66 244 2.83 
7 8.08 64 180 2.49 
8 5.62 626 1168 3.05 
9 2.09 60 58 1.43 

10 1.42 58 0 0 
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FIGURE 3 


CHARACTERISTIC VECTOR 1 


MEAN CROSS PRODUCT SOLUTION 
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component curves for each of the 10 trials. The results are given in 

both tabular (Table VII) and graphical (Figure 7) form. The last step 

in the unrotated solution was to calculate the factor scores for each 

individual on each component curve. The results are found in Table VIII. 
Since component curve analysis uses the model, 


Pe ey Poe tm Yo te Og 


ne Ties ene. M 


Le ki 
where un is the estimated score on trial j for person i, it is possible 
to obtain an estimate for any a from the Br matrix of component weights 
and the e. matrix of factor scores. For example, subject 1 on trial 1 
had a value of 19.9. Using values from row 1 of Table VII and row 1 of 
Table VIII, and substituting into the above equation, gives 

2G. 95) (07375) i+ X-12-52)C-0.293) + €-6.28)(-0.590) + CL OL (OF 133) e= 21 2. 
The discrepancy between the value of 21.2 and the actual value of 1929 is 
a result of using 4 components instead of 10 components to compute the 
estimate. In a similar manner it is possible to compute the complete X, 


A 
matrix and then form a residual matrix, Xx =X - X This difference 


k o 
represents one method of determining how well the first k components can 
reproduce the original data matrix. Instead of examining the complete re- 
sidual matrix, it is more desirable to refer to the mean Square ratio de- 
veloped by Tucker which compares the square of the k th eigenvalue with 
the sums of squares of the remaining eigenvalues. As explained earlier, 
this ratio provides an approximate statistical test for determining the 
number of components that should be retained. 

However the principle interest centers about two other focal 


points, namely the '"meaningfulness" of the component curves and their 


differentiating ability with respect to individual performances. The 
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TABLE VII 


COMPONENT CURVE ANALYSIS OF MEAN CROSS PRODUCT MATRIX 
LOADINGS OF TRIALS ON COMPONENT CURVES 
* 
(By MATRIX) 











Trial Component Curves 

'Year! 1 2 3 4 
13 36795 -12.52 -6,28 OL 
2 34.54 -10.03 0.30 -0,67 
3 32.88 - 6.68 4.04 -3.03 
4 32.40 - 2.00 4,13 -0.52 
2 32,02 0.60 2.79 Vas 
6 32320 4,22 1.05 32.01 
2 32,41 5.60 -0.08 2720 
8 325140 6.58 -0.91 Lab 
9 31.14 8.42 =2007 -2.30 
10 30.43 9.18 ~2.46 =35 31 
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TABLE VIII 


COMPONENT CURVE ANALYSIS OF MEAN CROSS PRODUCT MATRIX 
FACTOR SCORES ON UNROTATED COMPONENT CURVES 
KI 
(Y. MATRIX) 








k 
Subject Component Curves 
1D ul tah TUL IV 
1 OSS -0.293 -0.590 0.133 
# 0.608 -0.731 -2,223 -0.356 
3 1.446 -0,288 1.189 Os564 
4 0.487 OF S32 -0, 287 0.507 
5 1.196 -0.677 0.018 0.083 
6 Paw on -0.461 -0.957 0.348 
i 1 OLo -0.580 0.203 1.386 
8 0.392 -0.823 -1.146 0.005 
9 Pee -0.849 O72 75 0.158 
10 0.464 OA Oez7c -0.021 
ll O-707 -1,010 0.197 1.024 
Ws 0.255 -0.457 0.097 -0.171 
£3 i ee -0.477 0.850 -0.583 
14 Oa Er -0.737 -0.932 0.563 
16 0.256 =Zea og -0.113 0.002 
Op 1.828 1,798 OF O12 -1.650 
18 i al Bead 8 0.877 AP spe Hate) -0.064 
19 0.916 -0.210 0.485 0.671 
20 0.438 -1.456 -1.009 -0.901 
eal 1.029 OO. 126 -0.818 -0.699 
24 1.097 02333 0.254 Ov215 
25 Lua -0.215 0.438 OLOrs 
26 0.900 -1.709 -0,218 -2.373 
Jat j age els’ 0.374 -0.312 -0.850 
28 ig a -0,529 1.104 02574 
Si pen ES 1.298 -0.452 -1,126 
32 Ld 0.934 1,029 1,000 
So 0,349 -1,.891 -0.719 -1.909 
34 0.056 -0.274 =—OF127 -0.158 
a5 1 Alg -0,269 0.797 0.408 
36 1.074 -1,202 0.493 O;216 
37 0.606 1.246 -0.333 -0.176 
38 0.128 -0.362 0.589 -0.130 
39 Ee Fo 0,631 -0.025 -1,200 


40 0.059 -0.134 -0.668 G25 
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cc «eu, Ae 

s5e,0 ace it Ly oa 
ecc.0- gE ge kee 


Subject Component Curves 
1D I TT 1 ge IV 

ee er ee LAT et 
Al 0.756 -0.964 0.188 0.614 
42 0.504 -0.438 0.703 0.656 
43 1,228 =OF201 0.061 -0.549 
45 ay Rl -0.830 -0.026 -0.183 
46 0.398 -0.675 -0.050 0,234 
47 0.963 -0,9027 1.616 1.003 
48 1.306 1.190 0.672 0.269 
49 1.309 0.396 0.589 0.103 
51 1.388 =02073 0.702 0.197 
52 1.480 0.816 0.575 0.287 
53 1.499 0.900 0.820 0.062 
54 0.092 -0.578 -1.406 0.324 
55 0.646 =22733 0.842 -2.090 
56 1.658 OF 77 1 ms 20 -0.591 
57 1.589 1.681 -0.601 -0.324 
58 0.530 =0.325 -0.739 0.364 
59 0.010 -0.058 -0.083 -0.010 
60 0.038 SO G1.63 ~0.035 -0.159 
61 ros -0.856 1.047 1.193 
62 1355 1.345 JEEPS 0.551 
63 0.611 a1 h6S 0.568 0.645 
64 0.296 ap and ~2.398 0.444 
65 02273 -1.694 Shee 0.495 
66 1.245 0.682 0.131 0.317 
67 0.426 -0.476 ORG 17 e255 
68 1.395 0.165 507321 Omi35 
69 0.314 vis; -0.938 2.754 
70 1.099 ~0.846 -0.364 4,291 
72 0.902 334 1.618 -0.562 
73 0.673 -2.450 0.836 -0.936 
74 1.031 -0.091 -1.558 0.010 
7 he 1.667 0.847 =O, 224 = 1923 
N= 67 


TABLE VIII (Continued) 
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* 
first point may be investigated by examining the By matrix of factor 


* 
loadings; the second point requires focusing on the Y 


k matrix of factor 


scores. 

The nature of the unrotated component curves is easily visua- 
lized by referring to Figure 7. Component 1 is fairly close to the trial 
means. In this sense it may be said to represent the "average learning 
curve!’ - a concept which may or may not be of value. 

The remaining curves are a measure of the degree to which the 
average learning curve failed to account for the observed data. Thus the 
approximate average learning curve failed to represent a performance of 
strong increase, as shown by component 2 (or of strong decrease - if a 
person has a negative factor score on this component). For example, sub- 
jects 16, 55, and 73 have a low factor score on component 1 and a high 
(negative) factor score on component 2. A review of their actual per- 
formance (Table V) shows the close agreement between a description based 
on component curves and a description based on raw data. Similar state- 
ments may be made for components 3 and 4. 

It is possible to provide a verbal description for each of the 
components based on their shape. However it is important to note that 
each curve may be considered to represent two possibilities corresponding 
to the positive and negative factor scores on the components. Thus com- 
ponent 1 represents a slow drop in performance over the 10 trials, compo - 
nent 2 shows a strong steady increase (decrease) over the 10 trials, com- 
ponent 3 has a strong initial increase (decrease) for 3 trials followed 
by a gradual decrease (increase) for the remaining 7 trials, and compo - 
nent 4 shows a decrease (increase) for 3 trials, then an increase (de- 


crease) for 3 trials, followed by a gradual decrease (increase) over the 
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last 4 trials. Thus the first three components allow for a reasonably 
meaningful learning interpretation with the fourth component representing 
a more erratic performance. 

The second main point of interest focuses on the issue of in- 
dividual differences. The critical question appears to be, "To what 
extent may subjects be uniquely identified with a single (unrotated) 
component curve?". An examination of the factor scores on the component 
curves appears appropriate. One should note that a factor score on a 
particular component is independent of whether the component accounts for 
80% or 5% of the variance. The factor scores for each component curve 
are standardized with a mean of zero and standard deviation of one. Using 
a criterion that the factor score with the largest magnitude must be at 
least double the magnitude of the next largest factor score, a review of 
Table VIII suggests that subjects 5, 24, 43, 49, 56, and 68 are strongly 
associated with component 1; subjects 16, 37, and 73 with component 23 
subjects 2, 54, 62 and 65 with component 3; and subjects 67, 69, and 70 
with component 4. Therefore a total of 16 Sahaentee out of 67 or 24% of 
the subjects may be uniquely identified with a single component. Thus the 
unrotated solution does not appear to exhibit a strong simple structure 
in the person space. 

This finding may be the result of an interaction between the 
level of complexity of the task and the ability characteristics of the 
subjects. Thus if a perfect simple structure was observed in the factor 
score matrix for the four components, it would imply that the subjects 
may have used four clearly defined patterns of performance. The actual 
results show the untenability of this position - tastead there are a large 


number of possible patterns of performance, each pattern being approximated 
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by a set of four factor scores. 


oa 


Mean Cross Product Matrix - Orthogonal Rotations 


It is a property of factor analysis that there are an infinite 
number of solutions that may account for the observed data equally well. 
Thus it has become common practise to search for a method of rotation 
which will lead to some optimal "psychologically meaningful" solution. 
The same mathematical properties are true of component curve analysis. 

Any rotation may be expressed in matrix notation by the use of 


a transformation matrix, T, where 


-l1 _* ig 
dk you = YK ‘ 


k 
Note that any rotation does not change the estimates of the original data 


matrix, since 


eS) Ee 
ee Shay 
= B. tro} y 
ess ae: k 
pe: WARS 
= 5, Nae 


A general class of normalized orthogonal rotations is given by 


the following criterion: 


no 4 2 
by m n eh : 
pe Ss jp) -ws s jp _] = maximun, 
Ey pee j=1 -. 2 
j Bs 


where Te is the loading of variable j (j = 1, 2, ..., n) on the orthogo- 
nal factor p (p =1, 2, ..., m). There is evidence (Hakstian and Boyd, 
1970) to suggest that as the parameter w is increased, the variance dis- 


persion amon the factors tends to become more equal. The values of 0, l, 


m ; fe ' 
and /2 for w correspond to the well-known rotations quartimax, varimax, 
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and equamax. 


The three rotations quartimax, varimax, and equamax were applied 
to the factor score matrix re . The component curves corresponding to 
these rotations are given in Figures 8 - 10. Since these rotations are 
all designed to improve the simple structure of the input matrix, they 
represent a direct approach for improving the extent to which subjects 
may be uniquely identified with a single component curve. 

A review of an earlier section, Issue on Rotation of Axes, in- 
dicates that two criteria are to be used for evaluating the results of a 
particular rotation; the meaningfulness of the resulting curves based on 
a subjective evaluation of the smoothness and general shape of the rotated 
curves, and the increase in ability to distinguish among the subjects 
using the scores on the rotated curves. 

First consider the criterion of maximum separation among the 
subjects. All three rotations improved the simple structure in the per- 
son space to approximately the same degree. The unrotated solution iden- 
tified 16 students (24%) with a single component (criteria: magnitude of 
largest factor score must be more than twice the magnitude of the next 
largest factor score). After the rotations 37 students (55%) were uni- 
quely identified following the quartimax, 36 students (54%) after the 
varimax, and 38 students (57%) after the equamax. Thus the improvement 
toward simple structure was more than doubled resulting in slightly more 
than half the students being identified with a single component. 

The second criterion is related to the change in the shape of 
the component curves as a result of a rotation of axes. The following 
results deserve mention. After rotation the first component curve no 


longer approximates the trial means. The shapes of each of the three 
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rotated sets of curves have virtually no similarity to the shapes of the 
unrotated curves (Figure 7). The three sets of rotated curves, each set 
considered as a collection, have almost identically the same shape. How- 
ever although component curves three and four retain the same shape after 
all three rotations, the first component curve after the varimax rotation 
in the person space has the same shape as the second component curve after 
the quartimax or equamax rotation. Similarly, the second curve after the 
varimax rotation has the same shape as the first curve after the quarti- 
max Or equamax rotation. Generally, the rotated curves have a greater 
range of values than the unrotated curves. The rotated curves remain re- 
latively smooth and remain representative of a typical type of student 
performance. Thus students that can be uniquely identified with a rotated 
component curve tend to have an actual performance curve that is similar 
to the rotated curve. 

In conclusion, the orthomax rotations in the person space in- 
crease the extent to which students can be uniquely identified with a 
single component curve while maintaining the general meaningfulness asso- 
ciated with the shapes of the curves. All three rotations give virtually 


the same results. 


Data Matrix Minus Trial Means 

As mentioned in the earlier section, Issue on Choice of Matrix, 
the analysis of the data matrix with the trial means removed (i.e. the 
variance - covariance matrix) may be considered appropriate when the 
learning task remains the same and means are not an important parameter 


over the sequence of trials. The assumption of identical trials is war- 


ranted to the extent that one may assume that the general simulation 
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situation provides the major impact of the learning task, outweighing the 
influence of the cumulative specific values of the variables at a parti- 
cular trial. 

The relative value of applying Tucker's component curve me- 
thod to the variance - covariance matrix was evaluated by comparing the 
results of this analysis with those from the mean cross product solution. 
The same criteria used for evaluation of the rotated solution remain 
valid, namely the meaningfulness of the shape of the component curves 
and the ability to associate each student with a unique component curve. 
An additional criterion, the degree of relationship with an external set 
of measures, will be examined in a later section after the nature of the 
external measures has been described in greater detail. 

Since the trial means have been removed from the data matrix, 
this analysis focuses on relationships among deviation scores. Table IX 
gives the mean square ratios for the component curves that are extracted 
to account for the matrix of deviation scores. A comparison of these 
values with those of the F-distribution suggest that 4 components be re- 
tained. This is the same number that were obtained when the mean cross 
product matrix provided the initial data base. 

The loadings of the component curves for each of the 10 trials 
are given in Table X and are presented pictorially in Figure ll. A re- 
view of the unweighted characteristic vectors indicates very close agree- 
ment in the shapes of the curves from the variance - covariance matrix 
with those from the mean cross product matrix, although vectors two and 
four were reflected. Since the pattern of eigenvalues is different from 
the two solutions, the resulting pictures of the factor loadings obscures 


this similarity to some extent. 
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TABLE IX 


COMPONENT CURVE ANALYSIS OF VARIANCE - COVARIANCE MATRIX 
MEAN SQUARE RATIOS 
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TABLE X 


COMPONENT CURVE ANALYSIS OF VARIANCE - COVARIANCE MATRIX 
LOADINGS OF TRIALS ON COMPONENT CURVES 
* 
(By MATRIX) 


Trial Component Curves 
i 2 3 4 
1 8.90 9.02 -6.34 -1.66 
2 fie52 9.61 -0.82 0.36 
3 13.24 7.84 2202 ise'g)| 
4 yey ai! 4.23 5 ete Ona 
5 16c71 82 ZU -1.62 
6 16ch1 -1.80 oe. -3.45 
7 19.04 ee 0.26 -2.17 
8 {he eS -4.01 -0.46 -1.14 
a 19.80 -5.46 -1.94 PEA 
10 20.10 -5.89 -2.56 3.28 









; x Soeer | a 


AWAY, « AMR aRT 20 eo reviaite aay” 7 
2a ¥auo rane 10 = (AT WO Shr ieaos 
CxvareM qa a 


SSE es SS SS aS a Te 


23D tranbotos feist 
£ x ea - 


ll a 





FACTOR LOADINGS 


-3.00 1.00 2.00 9.00 13.00 1%.00 | 21.00 25.00 


7,00 


0.00 


2.00 


FIGURE 11 


UNROTATED COMPONENT CURVES 


VARIANCE COVARIANCE SOLUTION 


4.0 6.00 8.00 


0 
TRIALS 


10.00 


74 





12.00 


ae se ey og 9, | ee 
ce an rar. ian 


i - 





ho 


As before, factor scores were computed for each person on 
each component. Since the component curve solution provides an estimate 
forsthe deviation scores, it is possible to form a residual matrix of the 
difference between the actual deviation score and the estimate based on 
k components. For example, subject 1 had a value of 19.9 on trial 1. 

The mean value for trial 1 was 36.46 resulting in a deviation score of 


-16.56 for this subject. Using the component curve model 


X,, =b +Eb ee b 
ji 4: Yh j2 ert ‘ jk ee 


we obtain the estimate 


A 


X 
ll 


(8.90) (-.942)+(9.02)(-.715)+( -6.34)(-.020)+( -1.66)( .039) 


-14.76. 

However the principle advantage of using the variance - 
covariance matrix must be evaluated using other criteria - the "meaning - 
fulness" of the component curves and their usefulness in examining indi- 
vidual differences. 

Any interpretation of the shapes of the component curves 
should make explicit mention of the fact that the curves represent pat- 
terns of deviation scores. Thus the first component no longer represents 
an average learning curve. Instead it represents a dimension of maximum 
variability - subjects with a high score on component one are those whose 
performance showed increasing deviation from the mean performance for the 
group. For example, subject 56 has factor scores of 1.617, -0.187, -0.599, 
and 0.450 for the four respective components. The ten deviation scores 
were compuled: 2.5, 40,0 Pe bealgp lone: 20.0, AG sa .ekoee,. Liat, Liat, eld os 
Each of the succeeding curves represents other patterns of deviation scores 
- for example component 2 represents a decreasing deviation from the 


mean, ie. a gravitation toward the average performance. However since 
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the curve has negative loadings 6n the last 5 trials, this implies that 
this type of performance would represent continuing performance away 
from the mean, in a negative direction. Subject 65 has factor scores 
of -0.617, 2.417, 0.366, and 0.896. His deviation scores were 14.8, 
Pe?) 14075, 2.5,,0.4, -20.8, -2274, -23.6, -24:1, and =23,9, verifying 
this interpretation. 

Looking at the ability to discriminate individual differences, 
the unrotated factor score matrix was examined for simple structure. Re- 
call that 16 subjects (247%) were uniquely identified with one component 
from the unrotated mean cross product matrix, Using the same criterion, 
the unrotated variance - covariance matrix distinguished 18 such subjects 
(27%). There is no apparent improvement in simple structure. However 
this will be investigated further by examining the results of various 
rotations, 

The three rotations quartimax, varimax, and equamax were app- 
lied to the factor score matrix, The component curves corresponding to 
these rotations are given in Figures 12 - 14, The results were very 
similar to thos obtained when these rotations were applied to the mean 
cross product solution - the improvement an simple structure in the 
person space was more than doubled. Once again there appears to be no 


major difference between the results of the three rotations, 


Data Matrix Minus Person Means 

Since the subjects have different entering behaviors, and to 
the extent they exhibit a degree of consistency of performance over the 
task, where the n + 1 trial may depend on the n th trial, it may be con- 


sidered desirable to remove a portion of this effect from the performance 
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data by subtracting the individual subject means from each person's per- 
formance scores. Following this rationale, Tucker's component curve ana- 
lysis was applied to the data matrix with the person means removed. 

The mean square ratios (Table Xl) again indicate that four 
components be retained. The loadings of the component curves for each 
of the 10 trials are given in Table XII and Figure 15. A review of the 
shapes of these curves must take the nature of the analyzed data into 
consideration. The sharp negative slope of component curve one repre- 
sents a type of performance that begins with values well above a subject's 
individual average but which decreases over the ten values to a point 
well below the individual average. The curve is not a direct measure of 
how well the subject has performed on the simulation exercise, rather it 
is a measure of how well the subject has performed when compared to his 
individual average. For example, subjects 9 and 16 both had high scores 
on the first commence yet a review of Table 5 reveals subject 9 had an 
average score of 383 whereas subject 16 had an average score of 76. The 
second component curve indicates a performance that starts below person's 
average score, increases sharply over the next two trials to a value 
above the subject's personal average, followed by a gradual decrease in 
performance scores to a level slightly below the average score. Curves 
three and four have a much more noticeable oscillation about the subject's 
average performance and do not appear representative of a strong "learning" 
interpretation. 

A review of the factor scores on the unrotated curves permitted 
identification of 12 subjects (18%) with a unique component. This repre- 
sents a slight decrease from the 16 subjects that were equivalently iden- 


tified when the unrotated mean cross product solution was examined. 
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TABLE XI 


COMPONENT CURVE ANALYSIS OF DATA WITH PERSON 
MEANS REMOVED 


MEAN SQUARE RATIOS 
—?eeeeeee oo ————————————————_—_—______ 








Component Eigenvalue robmee df£2 Me an Square Ratio 


I 574.98 76 594 owe 
2 96.00 3 As 520 6.36 
3 52.40 72 448 6.07 
4 22°93" 70 378 4.02 
5 13,34 68 310 3.48 
6 8.30 66 244 3.34 
az 5.64 64 180 4.48 
8 2.10 62 118 yar | 
9 1,42 60 58 77.06 
10 0.02 58 0 0 
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TABLE XII 


COMPONENT CURVE ANALYSIS OF DATA WITH 
PERSON MEANS REMOVED 
LOADINGS OF TRIALS ON COMPONENT CURVES 








(By MATRIX) 
Trial Component Curve 
“ 1 2 3 a 
1 13.44 -6.41 0.76 -1.53 
2 10.56 0.36 -0.66 2.63 
= Gr 92 4,27 =2.87 G230 
4 2.24 4,16 -0.34 -2.19 
5 = 20,37 2./3 1.35 -1.89 
6 ay ee 0.77 3.70 Om> 
7 = Dots -0.42 2.3) Daeo 
8 - 6.19 =, 18 i325 1.42 
9 - 8.19 =2.08 =2.03 0.15 
10 = 9.08 =e -3.66 -0.88 
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FIGURE 15 
UNROTATED COMPONENT CURVES 
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Quartimax, varimax, and equamax rotations were applied to the 
factor score matrix. Only the quartimax results are given (Figure 16) 
since all three rotations yielded similar results. Thirty-three subjects 
(49%) were uniquely identifiable after the quartimax rotation was carried 
out, compared with 55% after the same rotation was carried out on the 


factor score matrix from the mean cross product solution. 


Double Centered Data Matrix 

The previous two sections explained the rationale for remov- 
ing either the trial means or the person means. Thus it is logically 
defensible to consider the effect of removing both sets of means - re- 
sulting in an analysis of the double centered data matrix. 

The results of removing the trial means from the data with 
the person means already removed was similar to the results of removing 
the trial means from the raw data matrix. In both cases the removal of 
the trial means did not materially affect the nature of the solution. The 
component curves resulting from an analysis of the double centered matrix 
are given in Figure 17 and are very similar to the curves obtained in the 
previous section (Figure 15). 

The efficacy of the solutions resulting from an analysis of 
the four types of data matrix (raw data, data minus trial means, data 
minus person means, and the double centered data matrix) will be investi- 
gated in greater detail when the relationship with a set of external mea- 


sures is considered in a later section. 
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FIGURE 16 
QUARTIMAX ROTATION APPLIED IN PERSON SPACE 
AND INVERSE IN TRIAL SPACE 
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FIGURE 17 


UNROTATED COMPONENT CURVES 


DOUBLE CENTERED DATA MATRIX 
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Il. COMPONENT CURVE ANALYSIS OF SIMULATED RANDOM PERFORMANCE 


There were two principle reasons for conducting a computer 
simulation of student performance. First, by randomly varying the inputs 
corresponding to the decision points in the simulation game FISHY, it was 
possible to actually test the game model before it was programmed for the 
IBM 1500 Instructional System and before the first students were used for 
further pre-testing. 

The similation of student decisions was repeated a number of 
times. The results of each run provided a basis for modifying both the 
initialization and the equations of the fishing model such that; (a) 
the magnitude of all results (both intermediate and final) was less than 
32,767, a constraint imposed by the limits of the IBM 1500 system, and 
(b) the final results were similar to those considered desirable assuming 
the student had made random decisions, namely that the value of the "cash 
on hand" variable showed a slight decrease over.the 10 trials, 

The second reason for conducting Monte Carlo studies on stu- 
dent performance data was to investigate the nature of the results of a 
component curve analysis on this type of data. This latter objective was 
achieved in two steps. First, a data matrix was generated without consi-- 
deration of the simulation game model, Thus a matrix with 100 rows and 
10 columns (corresponding to 100 people and 10 trials) was generated 
using the IBM subroutine RANDU (1968), so that each entry was obtained 
in a random manner from a rectangular distribution, The mean square ratios 
(Table XIII) clearly indicate that only one component curve should be 
retained. A review of the trial loadings indicated this curve is very 


close to the trial means. It may be inferred that the existence of more 
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TABLE XIIT 


COMPONENT CURVE ANALYSIS OF MATRIX OF 
. RANDOM NUMBERS 
MEAN SQUARE RATIOS 





Component Eigenvalue dfl df£2 Mean Square Ratio 


1 23.80 109 ~=891 30.04 
2 1.00 TO jeemalc 1.34 
3 96 105 679 1.38 
4 92 103 576 1.42 
5 75 101 475 1.24 
6 .68 99 376 1520 
7 62 97. 279 Tats 
8 . 60 95 184. 1a22 
9 .50 93 91 1.10 

10 45 91 0 0 
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than one component curve indicates the existence of something other than 
random data. It should also be noted that this does not imply that all 
non-random data will result in more than one component curve being iden- 
tified. 

The second step in the investigation of component curve ana- 
lysis involved imbedding the random number generation within the context 
of the fishing game model. The procedure involved generating a random 
number for each possible decision that could be made by a student at 
each trial. The only limitation imposed on the random values was that 
they remain within the range of admissible values. For example, the pro- 
cedure did not permit the catching of more fish than there were available. 
This was identical to the procedure used for originally testing the fish- 
ing game model except; (1) the final game model to be used in the study 
was specified at this stage, and (2) the results of the random perfor- 
mance were subjected to Tucker's component curve analysis. As before, a 
100 x 10 matrix of outcome values was penerated: The mean square ratios 
(Table XIV) indicate the retention of three component curves with the 
possible inclusion of the fourth component curve. The trial loadings 
(Table XV) are illustrated in Figure 18. 

The following points deserve mention. First, more than one 
component curve was required to account for the random performance. The 
structure of the game model appears to impose some structure on the re- 
sults of the component curve analysis. Second, the shapes of the compo- 
nent curves resulting from the analysis of random performance are very 
similar to those obtained from the actual student performance data 
(Figure 7). Two possible inferences may be made. The students, as a 


group, did not perform in a significantly different manner from a 
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TABLE XIV 


COMPONENT CURVE ANALYSIS OF MATRIX OF 
RANDOM PERFORMANCE 
MEAN SQUARE RATIOS 


eee 
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Component Eigenvalue dfl df2 Mean Square Ratio 


E 101.78 109 891 41.20 
2 Naya be. 107 784 95.54 
a 17,02 105 679 gs sds 
4 29 103 576 4,54 
5 re a) 101 475 3 LD 
6 04 99 376 1B Be: 
7, 04 a7 279 Ne 4 
8 .02 o&) 184 1.56 
9 .02 33 ot be 
10 SOE 91 0 0 
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TABLE XV 


COMPONENT CURVE ANALYSIS OF MATRIX 
OF RANDOM PERFORMANCE 
LOADINGS OF TRIALS ON COMPONENT CURVES 


* 
(BL MATRIX) 








Component Curve 

1 2 s 4 

4,19 -1.36 -0,.38 -O,15 
Siew =i p22 -0,26 -0.06 
3.44 -1.03 -0.14 0.06 
3.14 -0.83 0.06 0.16 
PS -0.49 0.29 O219 
2205 -0.04 0.46 0.08 
Zag 0.48 0.46 -0.08 
Psat fi 1.14 0.28 -0.22 
2.80 1.98 -0.04 -0.08 
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FIGURE 18 


UNROTATED COMPONENT CURVES 


RANDOM PERFORMANCE 
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Simulated set of random performance data. Alternatively, it may be true 
that component curve analysis tends to give similarly shaped curves under 
a variety of gig pepe te 

These results emphasize the importance of having a set of 
measures, independent of the data analyzed via component curve analysis, 
that can be related to the results of Tucker's analysis. One such set 


of external measures will be considered in the next section. 
III. FACTOR ANALYSIS OF MARKER TESTS FOR COGNITIVE FACTORS 


Two measures of "impulsivity - reflectivity" (mean latency 
time and number of errors) obtained from the computer-based MFF test were 
added to the ten recognized marker tests from the Kit of Reference Tests 
for Cognitive Factors (French, Ekstrom, and Price, 1967). These tests 
were hypothesized to represent a 6-factor space. A maximum likelihood 
procedure developed by Joreskog and described by Harman (p. 219) was used 
to obtain estimates of the factor loadings. 

Maximum likelihood factor analysis is based on the assum-: 
ption that the variables have a multivariate normal distribution. All 
twelve variables were tested for goodness of fit x2 statistic) to a nor- 
mal distribution. The two measures from the MFF test did not satisfy the 
normality criteria. Therefore, using a suggestion provided by Winer (p. 
218), these two variables were transformed to a new scale which better 
satisfies the assumptions underlying maximum likelihood factor analysis. 
Thus a natural logarithmic transformation was applied to the mean latency 
time and a square root transformation was used on the number of errors. 


The final results of the normality goodness-of-fit test are presented in 
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Table XVI. 

Maximum likelihood estimates of the factor loadings are deter- 
mined as the solution to two matrix equations. These equations cannot 
be solved algebraically, Syeeeed an iterative procedure must be used. 
Joreskog's (1967) computational method ensures the convergence of the 
iterative procedure. The method involves the removal, if necessary, of 
the variables that prevent convergence (this is called the improper, or 
Heywood case), obtaining a proper solution for the remaining variables, 
and then combining the previously removed variables with the proper solu- 
tion to obtain a final combined solution. If the unique variance of a 
variable approaches zero (i.e. the variable lies entirely in the common 
factor space), as the solution approaches convergence then it is possible 
to consider that variable as a factor. The variable (factor) is removed, 
the partial correlations are computed and the attempt at a proper conver- 
ged solution is repeated with the remaining variables. If such a Heywood 
case has arisen, the improper solution refers to the estimates of the fac- 
_tor loadings at the point where the 'special' variables fave zero unique 
variance (ie. before convergence has been attained). The corresponding 
proper solution refers to the loadings upon convergence of the maximum 
likelihood solution after these variables have been removed. A final 
“combined solution is obtained by incorporating the previously removed 
variables into the proper solution. The inclusion of a variable (factor) 
to the proper solution involves the addition of one row and one column 
to the factor loading matrix. Since the removed variable may be consi- 
dered an orthogonal factor, the entries for the new row are all zero 
except for a one where the row intersects with the corresponding new 


column. The column entries consist of the original correlations of the 
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TABLE XVI 


MARKER TESTS FOR STUDENTS' CHARACTERISTICS 
GOODNESS OF FIT TEST TO NORMAL DISTRIBUTION 


Variable (Marker Test) 


oman aw FWD re 


Letter Sets (1-1) 

Figure Classification (1-3) 

Object - Number (Ma-2) 

First and Last Names (Ma-3) 

Addition Test (N-1) 

Subtraction and Multiplication (N-3) 
Mathematics Aptitude (R-1) 

Necessary Arithmetic Operations (R-4) 
Vocabulary Test (V-1) 

Vocabulary Test (V-2) 

MFF - No. of Errors 

MFF - Mean Latency Time 


11' -MFF - Square Root (No. of Errors) 
12' -MFF - Log, (Mean Latency Time) 
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retained variables with this new factor. This process is repeated until 
the complete set of variables is included in the final solution. A de- 
tailed explanation of the procedure may be found in an article by Joreskog 
(1967). The maximum likelihood approach includes a test for goodness of 
fit that was initially developed by Lawley (Joreskog, 1967). This test 
may be used to determine the appropriate number of common factors. The 
procedure is outlined by Harman (p. 220). 

Analysis of the marker test scores revealed an improper solu- 
tion. The goodness of fit test indicated 5 factors should be retained. 
The initial improper solution using all twelve variables is given in Table 
XVII. Since variables 3, 6, 7, and 9 had no remaining unique variance, 
the previously mentioned adjustment was made. The final unrotated solu- 
tion is presented in Table XVIII. 

The selection of a criterion for rotation was based on the fol- 
lowing considerations. First, improvement toward simple structure was 
desirable to permit an easier identification and interpretation of the 
obtained factors. Second, since the initial battery of marker tests were 
designed to identify independent factors, an orthogonal rotation was con- 
sidered necessary. The possibilities were thus reduced to the family of 
orthomax rotations. Since each factor was based on an equal number of 
marker tests, it was desirable to distribute the variance equally over 
all five factors. Therefore an equamax rotation was selected as the final 
step prior to identification of the factors. The results of this rotation 
are provided in Table XIX. 

An examination of these loadings permits the following inter- 


pretation for each factor: 
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TABLE XVII 


MAXIMUM LIKELIHOOD FACTOR ANALYSIS OF MARKER TESTS 
IMPROPER SOLUTION 


Marker Test i LL paw IV 
1 (1-1) Induction eyo by 087 149 -,107 
2 (1-3) Induction e255 144 63 .104 
3 (Ma-2) Memory . 606 -.194 a Ps! ~207 
4 (Ma-3) Memory ey ey <auD .090 
5 (N-1) Number Facility .655 -.471 Pisse) e205 
6 (N-3) Number Facility .622 -.630 +2300) “5500 
7 (R-1) Gen.Reasoning e733 ~ 136 451 469 
8 (R-4) Gen.Reasoning . 648 v2L2 i335 eatsie) 
9 (V-1) Verbal Comp. Beore coop | 2 O81, =) Sea 
10 (V-2) Verbal Comp. Toba . 509 -044 -.,266 
id? Sqekt, (MEF.Error) = .2a4 -.2/0 £203 s053 
12' LN (MFF Latency) <oee 5202 rg BY as - 168 
pe Bea oeileo2s: 1, 201- 0, 772 
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MAXIMUM LIKELIHOOD FACTOR ANALYSIS OF MARKER TESTS 


TABLE XVIII 


FINAL UNROTATED SOLUTION 


[OO 


Marker Test 


(1-1) Induction 

(1-3) Induction 
(Ma-2) Memory 

(Ma-3) Memory 

(N-1) Number Facility 
(N-3) Number Facility 
(R-1) Gen.Reasoning 
(R-4) Gen.Reasoning 
(V-1) Verbal Comp. 

10 (V-2) Verbal Comp. 
11? Sq.Rt.(MFF Error) 

12' Ln (MFF Latency) 


Seas 


OMAN AU FPWNHeE 


-.489 
~.281 
-.624 
-.554 
-.648 
-.620 
-. 736 
-.643 
-.982 
-.376 

=218 
-.298 


S00 


083 
a ey 
-,188 
i 2D 
-.460 
-,631 
lcs 7 
.206 
714 
. 504 
-.245 
,1 82 


1.610 


pLa7 
3 G7 
-./29 
- .380 
4133 
4233 
477 
144 
-,070 
048 
=, 19% 
kel 


A py 





-.112 
~095 
2b 1 
oO 2 

-.280 

-,403 
464 
2b24 

-.384 

-.260 
Ose 
«167 
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MAXIMUM LIKELIHOOD FACTOR ANALYSIS OF MARKER TESTS 


TABLE XIX 


FACTOR LOADINGS AFTER EQUAMAX ROTATION 


— 


Marker Test 


(I-1) Induction 

(1-3) Induction 
(Ma-2) Memory 

(Ma-3) Memory 

(N-1) Number Facility 
(N-3) Number Facility 
(R-1) Gen.Reasoning 
(R-4) Gen.Reasoning 
(V-1) Verbal Comp. 

10 (V-2) Verbal Comp. 
11' Sq.Rt.(MFF Error) 

12' Ln (MFF Latency) 


< a 


OMAHA UKWNrH 


-13/ 
-.087 
981 
629 
2299 
zO7> 
214 
296 
. 234 
2079 
7031 
eo 


1,644 


Le 


-2g8 
. 140 
-.025 
ay. 
»050 
-.019 
079 
2002 
~951 
. 638 
-.219 
093 


1,628 
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5308 
. 100 
184 
. 066 
.813 
2992 
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Factor I - Associative (Rote) Memory 
Factor II - Verbal Comprehension 
Factor III - Number Facility 

Factor IV - General Reasoning 

Factor V - Impulsivity 


The first four factors coincide with those provided in the Manual for Kit 
of Reference Tests for Cognitive Factors (French, Ekstrom, and peices 
1963). Factor V - Impulsivity - is based on the high positive loading 
for the variable "square root of the number of errors on the MFF test" 
and the high negative loading for the variable "natural logarithm of the 
average response latency time''. These factors will be utilized in the 
last phase of this ecude which will examine relationships between these 
characteristics of the subjects and their performance on the simulation 
game as described by the component curves. 

Before these relationships can be investigated it is necessary 
to compute the factor scores for each subject on each of the five identi- 
fied factors. Harris (1967) has reviewed four methods of computing such 
factor scores. Harris adopts the conventional definition of "true" fac- 
tor scores as direction cosines locating the common factors in the person 
Space. One of the methods reviewed is Bartlett's estimate of the factor 
scores. The matrix equation for this method is 

F=-zu*acatu’ ay? 

where Z is the N- by -n matrix of standard scores 
v2 is the m- by -m diagonal matrix of unique variances 
A is the n- by -m matrix of factor pattern coefficients 
F is the N- by -m matrix of estimated factor scores. 


This estimate is univocal, that is, the correlation matrix between F and 
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the "true" factor scores is the identity matrix. Anderson and Rubin 


(Harris, 1967, p. 372) have developed a method that insures that F F =I, 
but for an orthogonal rotation (for example, equamax) these two methods 
provide proportional factor scores. 

Bartlett's method was used to obtain the factor scores for the 
students on the five equamax rotated cognitive factors thereby maintain- 
ing the univocal nature of the results (Table XX). The intercorrelation 
matrix fF given in Table XXI provides a good approximation to the iden- 
tity matrix indicating the essential orthogonal nature of the factor 
scores. 

The last two sections have reported the results of the students! 
performance (via component curve analysis) on a simulation game oil OWS 
scores on an independent set of measures (via factor analysis) related to 


Cognitive abilities. The following section will examine the relationships 


between these two sets of data. 


IV. RELATIONSHIP BETWEEN COMPONENT CURVES AND 
EXTERNAL MEASURES OF SUBJECTS' CHARACTERISTICS 


Canonical Correlation Approach 


Canonical correlation methods involve determining a set of 
weights for two sets of variables such that the multiple correlation be- 
tween the two weighted sets is a maximum. If one of the sets contains a 
single variable the method becomes one of multiple regression (as used in 
the next section). Several sets of weights are frequently possible, sub- 
ject to the restriction that they be independent of other sets of weights. 


Bartlett (Cooley and Lohnes, p. 37) has developed a chi-square test for 
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TABLE XX 


MAXIMUM LIKELIHOOD FACTOR ANALYSIS OF MARKER TESTS 
FACTOR SCORES ON ROTATED (EQUAMAX) FACTORS 


Subject 


I 


-1,.198 
-0.329 
-0.482 
-0.346 
0.382 


-0.753 
-1.301 

0.128 
-0,068 
-0.453 


= 293 
-1.596 
-0.778 
0.507 
-1.848 


-0.432 
-0,105 
0.388 
0.963 
-1,294 


-0.063 
-0,. 833 
-0.283 
-0. 700 

1.417 


0.631 
1.764 
1.618 
-0.425 
Pa a Me 


-1,254 
-0.726 
-0.034 
-0.022 
-0.552 


FACTOR 

fist Tee 
-0.057 1.160 
0.691 -0.462 
0.034 -0.084 
-0.944 -1.640 
-0.630 -0.505 
OD bE -0,842 
-0.365 -0.244 
0.045 -0.134 
Vea -1.419 
0,697 We675 
-1.425 -0.297 
-0,284 0.026 
-0.060 On673 
| cy Ligh b= 
-0.844 1,402 
0.378 2.840 
-0.686 -0.748 
-0.578 0.714 
-0.030 -0.811 
-0.198 -0.084 
-0.312 -0.468 
0.461 = Ree 
1.421 -0.630 
-1,253 -1.214 
=O. 255 Hie Sas 
-0.796 ee 
-0.800 -0.874 
-0.612 0.150 
0.448 -0.932 
1.689 Oe f 
-1.407 0.807 
Ge079 -0.005 
-0.484 0.962 
1.462 -0.676 


-1.410 


0.333 


IV 


1,286 
SHS ey 
-0. 204 
0.306 
-0. 706 


-0.683 
0.687 
-0.612 
— tee 
-1,289 


0.984 
-0.961 
-1,015 

0.108 
-1.472 


3.348 
-0.778 
0.620 
-0.458 
-0.570 


02970 
-0.249 
nOe2o2 
-0.198 

0.183 


-1,194 
-0.280 
-0.333 

0.106 
-0.886 


-0.696 
-0,273 
-0.659 
0.448 
0.601 
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TABLE .XX (Continued) 


— Eee 





Subject FACTOR 
mee cl. I Le LE IV V 
41 2.063 -2.099 -0.105 -0.813 0.169 
42 -0.517 0.938 0.349 On OF -0.174 
43 Berg -0.891 -1.633 0.809 22062 
45 -0.451 0.658 -0.630 1.190 0.957 
46 -1.391 0.586 je Ee -0.585 -0.235 
47 1.548 el a2 -0.754 12570 Os 17, 
48 -1,073 -0.958 -1.679 -0.183 0.971 
49 fees yl -1.110 Ont le -0.874 Ors tS 
aus 07425 -0.663 -0.345 -0.038 -1.065 
BY; 1-476 0.029 -0.847 0.003 -0.048 
oe 0.698 2.289 -0.780 -0.564 eee L 
54 -0.688 -0.852 0.078 -0.312 OF227 
55 -0. 860 -0.361 0.868 O.727 ae ee hs ek 
56 17637 -1.610 -0.231 2.024 -2.135 
57 .-0,266 0.234 0.619 0.751 -0.443 
58 Oeki2 -0.755 1.014 0.237 0.976 
59 -1,538 0.764 0.493 02259 0.614 
60 0.748 2.890 i226 -0.449 -0.913 
61 -0.703 O22) -1.098 0.534 0.846 
62 -0.264 0.958 -0.548 1.400 -0.606 
63 1379 -0.320 2. 2aS -0.249 1.387 
64 -0.011 ey i -1.476 -1,037 1.524 
65 -0,390 -0. 738 Os158 -0.838 -1.628 
66 0.076 -0.746 -0.449 0.386 -1,.888 
67 Lugs -0.024 1.450 -0.802 0.998 
68 0.865 -0.265 O27 76 0.784 85] 
69 17443 OFZ7r -0,725 0.535 13307 
70 -0.285 -0.154 -0.725 -1.460 -0.410 
dee -0.659 0.884 -0.458 O8b72 OF3l7 
75s) =O2 221 2.339 0.873 -0.557 0.410 
74 -0.554 0.898 whee Led so -0.050 
fps O723/] -1,092 Mead Ae -0.350 -2.234 
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the null hypothesis that the two sets of variables are unrelated. 

A canonical correlation analysis was carried out for the scores 
On the unrotated component curves from the cross-product solution and the 
factor scores on the set of 5 cognitive factors. The input correlation 
matrix is presented in Table XXI and the results are given in Table XXII. 
The maximum canonical correlation was 0.57 which is significant at the 
0.05 level. There was only one set of weights that yielded a significant 
canonical correlation. 

The weighting system indicates that component curve I is the 
most important component in the sense of being maximally related to the 
set of cognitive factor scores. Similarly, the most important cognitive 
factors are impulsivity, general reasoning, and number facility. 

The canonical analysis was repreated for the various rotated 
solutions (Table XXIII). The results of these additional analyses may be 
summarized as follows: (1) the canonical correlation for the various ro- 
tated solutions does not change as would be expected since these solutions 
are all transformations of one another, (2) the weights for the cognitive 
factors remain constant, (3) the interchange of the weights between curves 
I and II when changing from a quartimax to a varimax rotation in the per- 
son space agrees with the interchange of shapes of these two component 
curves as seen in Figures 8 and 9. The canonical weighting system is 
primarily dependent upon the shape of the curves, and (4) with the excep- 
tion of the previous point, there is no appreciable difference in the 
weights among the three rotated solutions, just as there is no appreciable 
difference among the shapes of the curves from the three solutions. 

A comparison of the results of the canonical correlation be- 


tween the unrotated curves (Figure 7) and the curves after the quartimax 
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TABLE XXI1 


RESULTS OF CANONICAL CORRELATION OF FOUR COMPONENT 
CURVES WITH FIVE COGNITIVE FACTORS 


SSS 


Component Curves Cognitive Factors 
ee ee ee ee ee ee) ee 
us 899 1. Assoc. Memory -.345 
LL 187 2. Verbal Comp. -.238 
TLL -.299 3. Number Facil. -.442 
IV -202 4. Gen.Reasoning 455 

5. Impulsivity -.650 
a Fe VRS 6 Ra 

Ri mee Osa, wee Sas pe BO p <0.05 
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TABLE XXIII 


RESULTS OF CANONICAL CORRELATION OF FOUR 
ROTATED COMPONENT CURVES WITH FIVE COGNITIVE FACTORS 








a) QUARTIMAX IN PERSON SPACE 





Component Curves Cognitive Factors 
r -.053 1. Assoc. Memory -.345 
Ey 504 2. Verbal Comp. -.238 
ie wit 2205 3. Number Facil. -.442 
IV -398 4. Gen.Reasoning 454 

5. Impulsivity -.650 
a Petes; ee 
Cow y hee Gee 3.51% p<0.05 





b) VARIMAX IN PERSON SPACE 





Component Curves Cognitive Factors 
1 -500 1. Assoc. Memory -.345 
pat -.036 2. Verbal -.238 
fig «Ah OF 3. Number Facil. -.442 
IV -401 4. Gen.Reasoning ~454 
5. Impulsivity -.650 
2 
R, = 0.57, xe 69 G033.51, p<0.05 





c) EQUAMAX IN PERSON SPACE 





Component Curves Cognitive Factors 
I -.020 1. Assoc. Memory -.345 
ims 497 2. Verbal -.238 
Lit .769 3. Number Facil. -.442 
IV -403 4. Gen.Reasoning ~454 
5. Impulsivity -.650 
z 


Rye = 0057 5 X 49 OS dD p< 0.05 
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rotation in the person space (Figure 8) indicates the general spread of 
importance (in the sense of higher weights) following the rotation. Thus 
Component curve I (unrotated) had a weight of .899 followed by curve III 
(-.299) whereas curve III (rotated) had a weight of .765, curve II (.504); 
and curve IV (.398). 

Canonical correlations were also computed for the other three 
types of component curve solution. These results can be summarized 
briefly. The removal of the trial means had virtually no effect on either 
the canonical correlation or the weighting system. Once again, this is 
in agreement with the minor change in the shapes of the curves. The re- 
moval of the person means reduced the canonical correlation to a level of 
Statistical non-significance, therefore no weights are reported. Similar 
results were noted for the double-centered matrix. 

In addition to revealing the relationships between the component 
curve solution and a set of ability measures, canonical correlation may 
be used to evaluate the relative efficiency of the solution as compared 
with the original raw data. Data reduction was achieved in two ways: the 
ten trials were reduced to four component curves and the twelve marker 
tests were reduced to five cognitive factors. Referring to Table XXIV, 
comparison of the results of a canonical correlation analysis of both 
sets of raw data with both sets of reduced data reveals that a total data 
reduction of 59% results in a loss in percent of variance accounted for 
of 30%. This implies that combined use of component curve analysis and 
factor analysis resulted in a descriptive system approximately twice as 
efficient as one using the actual raw data. 

The relationship between the cognitive factors and the scores on 


the component curves will be investigated in greater detail in the following 
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TABLE XXIV 


COMPARISON OF EFFICIENCY OF 
DATA REDUCTION METHODS 








bh of 
Method max.R, % Variance Total no. original 
accounted for of scores Total 
ar ee ee ee Se CtOT Veriavies (fentor scormst on 
ie Raw performance scores 
+ marker test scores 79 62 1474 100 
at Raw performance scores 
+ cog.factor scores ~/0 49 1005 68 
3. Comp. curve scores 
+ marker test scores 64 41 1072 73 
4, Comp. curve scores 
+ cog.factor scores iF 32 603 4l 
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section, where the cognitive factor scores will be compared with the in- 
dividual component curves, rather than with the set of curves as was done 


in this section. 


Multiple Linear Regression Approach 
The purpose of the multiple regression method is to derive 
weights ee (3 as ( for the predictor variables (factor scores on 


SG eee Ne 


the cognitive factors) X,, X,, 


5? and an additive constant (&. such 


that the resulting weighted composite variable Y, defined by 
= a X ah PENS 
ae (Bet Gi 5 %5 
predicts a criterion variable, Y with a minimum sum of squared error. 
The present situation has one notable feature - the predictor 


variables are all independent of one another. This results in a consider- 


able simplication of the usual formulae; for example, it now follows that 


“~» 
H} 

KH 

vp) 


ale >)? See 
5x 
2 2 2 Zz 
and R =r =r he ie ea 
YX, YX YX5 ; 


Darlington (1968) discusses three principle measures of the "importance" 
of a predictor variable: (1) the squared validity, er 3 (2) the stan- 
dardized beta weight, @ 3; (3) the usefulness of ny? ee as the 
amount R? would drop if AG were removed from the regression equation; and 
points out that these are all equivalent in the special case of uncorre- 
lated predictor variables. However the sign of the beta weight provides 
additional information - it indicates the direction of the relationship 


and may thus be used to help identify the nature of the criterion vari- 


able with respect to the nature of the predictor variable. 
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Draper and Smith (p. 171) describe a stepwise regression pro- 
cedure which gives the values of the F-statistic for testing the contri- 
bution of each predictor variable to the existing regression equation. A 
computer program (MULRO6: Div. of Educational Research services, U. of 
Alberta) based on this procedure was used to perform the necessary regres- 
sion analyses. 

The factor scores on a particular component curve were taken 
as the criterion variable and the factor scores on the five cognitive 
factors were used as the predictor variables. Thus for a specified com- 
ponent curve solution, say the unrotated cross-product solution, a total 
of four regression equations were derived, one for each component curve. 

Table XXV gives the standardized beta weights for all signifi- 
cant (%= 0.10) predictors for the unrotated component curves. The mag - 
nitude of these beta weights provide a measure of the importance of the 
various predictors, It is informative to refer to the shapes of the res- 
pective curves (Figure 7) when describing the relationship between the 
component curves and their predictors. 

Consider the unrotated cross-product solution. Component I re- 
presents fairly constant performance over the 10 eae of the simulation 
game. Three factors; associative (rote) memory, general reasoning, and 
impulsivity are related to scores on this component. Students who have 
high scores on memory and general reasoning and a low score on impulsi- 
vity tend to have higher scores on this component curve. For example; 
subject 56 had a high loading on component I (See Table VIII) his stan- 
dardized scores on the three cognitive factors were: associative (rate) 
memory, 1.637; general reasoning, 2.024, and impulsivity, -2.135. For 


component II, which shows a strong improvement in performance, the 
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TABLE XXV 


STANDARDIZED BETA WEIGHTS FOR SIGNIFICANT! PREDICTORS 


OF FACTOR SCORES ON UNROTATED COMPONENT CURVES 








PREDICTORS CROSS-PRODUCT SOLUTION 
Component 

I If Tit IV 
Ly Assoc .Memory ~ 23 - ~ - 
va Verbal Comp. - ~ - - 
Sie Number Facility - - ~ -.26 
4. Gen.Reasoning mee e220 + - 
ee Impulsivity -.36 ~eae - - 
% Variance 22.06 16.0 - (si 7 


Accounted for 


1. Level of significance = 0.10 
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memory factor is no longer associated with performance. The general 
reasoning factor shows a slightly stronger relationship with scores on 
this component and impulsivity shows a Slightly weaker relationship. 
Subject 16 provides a good illustration of this situation. This subject 
had a high negative score on component Il, therefore one would expect 
this person to have a low score on general reasoning and a high score 

on impulsivity. The standardized factor scores are -1.472 and 0.706 res- 
pectively. These statements are illustrated in Figure 19 which gives 
selected learning curves for four students, including subjects 56 and 16 
mentioned above. 

Another interesting feature is brought out by noting the nega- 
tive relationship between number facility and component IV. At first 
glance this is a disturbing finding - why should an ability such as number 
facility hinder performance? However it is an easy manner to reflect this 
component, thereby bringing the results into agreement with intuition. 
Therefore the set of beta weights may not only be used to help identify 
components but may also be used to guide one particular type of rotation, 
reflection (a rotation of 180°). 

It was mentioned in the previous section, Mean Cross Product 
Matrix - Orthogonal Rotations, that different rotations may result in si- 
milar shaped component curves. For example, component I following a quar - 
timax rotation in the person space (Figure 8) has virtually an identical 
shape to component II after a varimax rotation in the person space (Figure 
9). The standardized beta weights in Table XXVI are almost the same for 
these two components. 

The regression analyses revealed a lack of improvement in pre- 


diction among the different rotated solutions. In fact there was a slight 
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(The dots represent actual performance scores x 107) 
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Subject 56 
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decrease in the amount of predictability (% of variance accounted for) 
for the rotated solutions. Thus it appears that the transformations re- 
sult in more extreme shapes for the component curves - curves which in 
turn bear less relationship to a set of measures on cognitive abilities. 
The next chapter will summarize the results obtained from the 
component curve analyses of the student performance data on the simula- 
tion game, the maximum likelihood factor analysis of the marker tests for 
the cognitive factors, the canonical correlation between the two result- 
ing sets of data, and the multiple linear regression of the cognitive 
factor scores on the component curve scores. Conclusions based on these 
results will be stated and suggestions for further research will be pre- 


sented, 
Wen SUBJECTS! REACTION TO THE STUDY 


The Final step in the collection of data required the students 
to indicate their position on a 7-point scale (1 = dislike, 7 = like) for 
the following two statements: 

1. Indicate on the pais your feelings about taking part of your 

school curriculum on a computer terminal. 

2. Indicate on the scale your feelings about the game FISHY. 
The mean scale values were 6.8 and 6.2 respectively for these two state- 
ments. These two questions were asked approximately one week after their 
participation on the computer terminals. It seems therefore that the stu- 
dents had a generally favorable attitude about both computer-assisted in- 
struction and the type of simulation game represented by FISHY. 


Approximately two-thirds of the subjects felt the set of 
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cognitive tests were either too hard or too long. The complete set of 
tests took approximately 2 hours and 20 minutes to complete. The recom- 


mended minimum grade level for these tests was either grade 6 or grade 8. 
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CHAPTER V 
SUMMARY, CONCLUSIONS, AND RECOMMENDATIONS 


I. SUMMARY 

The primary purpose of this study was to evaluate Tucker's com- 
ponent curve analysis as a method for describing student performance on 
a computer-based simulation game. The study focused on the characteris- 
tics of the component curves resulting from variations in applying the 
method of component curve analysis to the student performance data. 

The data base for the study consisted of a random sample of 67 
grade eight students from one junior high school in Edmonton, Alberta. 
The students were introduced to the computer instructional staat ett 
then they proceeded through ten cycles of a simulation game on the sal- 
mon fishing industry. The scores for each student consisted of the 
value of the "cash on hand" variable at the end of each cycle. After 
the students had completed the simulation exercise they received a com- 
puter based version of the Matching Familiar Figures test. This test 
provided a measure on an impulsive-reflective dimension. The following 
week a set of 10 marker tests from the Kit of Reference Tests for Cog- 
nitive Factors (French, Ekstrom, and Price, 1967) were administered to 
the students. 

The data collected by the above procedure was then subjected to 
the following analyses. First, a component curve analysis of the stu- 
dent performance data on the simulation game was performed. Four com- 
ponent curves were identified and factor scores on these four curves 


were computed. The shape of the curves represented typical performance, 
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in the sense that subjects with high scores on a particular component 
tended to have an actual performance curve with a shape similar to that 
of the curve in question. Using a criterion that the factor score with 
the largest magnitude must be at ee double the magnitude of the next 
largest factor score, 24% of the subjects were uniquely identified with 
a single component curve. The solution was then subjected to three ro- 
tations in the person space: quartimax, varimax, and equamax. All three 
rotations gave virtually the same result. The percentage of students 
identifiable with a unique component increased to about 55% while the 
general shapes of the curves remained representative of typical student 
performance. 

Second, a component curve analysis of the data with the trial 
means removed was performed. This procedure was considered appropriate 
to the extent that one could consider the simulation task as remaining 
the same over the series of trials. The results of this analysis were 
very similar to the previous results. The interpretation of the shapes 
of the component curves emphasized that the curves represented patterns 
of deviation scores. 

Third, a component curve analysis of the data with the person 
means removed was carried out. Ties Guelied analyzing all subjects on 
the same base line - performance about their own means. The resulting 
curves did not permit as strong a learning interpretation as the other 
two solutions. The results of the three rotations in the person space 
increased the percentage of students identifiable with a unique curve to 
about 49%. 

Combining the rationale of the previous two analyses, a compo- 


nent curve analysis of the double centered matrix was then conducted. 
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The results of this analysis were very similar to the results from the 
data with only the person means removed - the additional removal of the 
trial means did not noticeably modify the results. 

Two additional component curve analyses were conducted on sets 
of random data. First, a matrix of random numbers was analyzed. The 
results clearly indicated that one component curve could describe the 
data satisfactorily. Second, random numbers were used to simulate stu- 
dent decisions at each decision point in the fishing game model. A com- 
ponent curve analysis of the resulting performance data indicated the 
retention of three component curves. The structure of the game model 
imposed some structure on the nature of the results of the component 
curve analysis. The shapes of these three curves were similar to those 
obtained from the original component curve analysis of the actual student 
performance data. 

The next step consisted of performing a maximum likelihood fac- 
tor analysis on the ten marker tests plus the two measures of impulsivity 
obtained from the Matching Familiar Figures test. Five factors were se- 
lected and subjected to an equamax rotation. The resulting five factors 
were identified as Associative (Rote) Memory, Verbal Comprehension, Number 
Facility, General Reasoning, and Impulsivity. Factor scores on these 
factors were computed for each student. 

The relationship between this set of cognitive factor scores and 
the scores on the component curves was then examined by means of canonical 
correlation analyses. The largest weights were assigned to the first un- 
rotated component curve and to the cognitive factors impulsivity, general 
reasoning, and number facility. When the method was applied to scores 


on the curves after the quartimax rotation, the system of weights was 
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more even for the component curves although the canonical correlation of 
0.57 was not changed. A comparison of the proportion of data reduction 
(via component curve analysis and maximum likelihood factor analysis) 
versus a loss in canonical correlation indicated that a total data re- 
duction of 59% resulted in a loss of percent of variance accounted for 
of 30%. 

Finally, a more detailed analysis of the relationship between 
the cognitive factor scores and the scores on the component curves was 
conducted by means of multiple linear regression. Referring to the un- 
rotated component curves, the scores on curve I were significantly re- 
lated to the scores on associative (rote) memory, general reasoning, and 
impulsivity; curve II was related to general reasoning and impulsivity; 
and curve IV was related to number facility. After the quartimax rota- 
tion curve I was significantly related to number facility and curve IIL 
was related to general reasoning and impulsivity. 

The next section presents the conclusions arising from this 


study. 


II. CONCLUSIONS 


In general, the results of this study have indicated the poten- 
tial value of Tucker's component curve analysis for describing student 
performance on a complex simulation game. Since more than one component 
curve was required to account for the variation in the performance data, 
the method possesses the important advantage of explicitly delineating 
individual differences in performance when they do occur. This feature 


was shown to be related to individual differences in cognitive abilities. 
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The results of the study suggest the value of analyzing the 
mean cross product matrix in the component curve analysis. This proce- 
dure utilizes all of the available information in the analysis. The re- 
moval of the trial means did not appreciably alter the results. This may 
be explained by the similarity of the means over the 10 trials. The effect 
was primarily one of subtracting a constant from the data matrix, result- 
ing in only a slight change in the nature of the results. In contrast to 
this, the removal of the person means had a noticeable effect on the re- 
sults. But this resulted in the removal of the main source of correlation 
between scores on the component curves and scores on the cognitive factors. 
The results of a component curve analysis of the double centered matrix 
were similar to those resulting from the analysis of the data with the 
person means removed. 

The three methods of rotation provided mixed results. All three 
rotations were essentially equivalent, therefore only the quartimax rota- 
tion in the person space will be mentioned. The principal advantage from 
the rotation was the approximate doubling of the number of subjects that 
could be uniquely identified with Beri cic component. Although this ro- 
tation did not affect the canonical correlation with the scores on the 
cognitive factors, it did reduce the number of statistically significant 
relationships between the cognitive factors and the individual component 
curves, as examined by means of multiple linear regression. 

An unexpected finding, but one that may have major implications, 
was noticed when the random performance data was analyzed by means of 
component curve analysis. Three component curves were required to account 
for the data instead of only one curve. The structure of the simulation 


game model had imposed a structure on the component curve solution. This 
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may thus provide a method for classifying such games. 

This same analysis emphasized the importance of having a set 
of independent measures to provide additional information on the nature 
of the component curve solution. The shapes of the three component curves 
derived from the random performance were very similar to the shapes of 
the curves derived from the actual student performance. Thus an inter- 
pretation based on the relationship between the scores on the component 
curves and the scores on the cognitive factors was necessary to ensure 
their value. 

The results of the canonical correlation analyses revealed 
two other points that deserve comment. First, the importance of the com- 
ponent curves as measured by the magnitude of the weights for the respec- 
tive component curves did not agree with the importance of the etree as 
measured by their eigenvalues. For example, component II (unrotated) had 
a smaller weight than either curves III or IV (Table XXII). This may 
imply the omission of the some relevant factors, such as short term me- 
mory, from the set of external measures. Second, there was a net increase 
in efficiency resulting from the gain measured in terms of data reduction 
versus the loss in precision of the relation between the scores on the 
component curves and the scores on the cognitive factors. 

The use of simulation games and models as an instructional 
technique is a new and expanding development in education today. Tucker's 
component curve analysis represents a method for describing student per- 
formance on such a learning situation. The technique has the potential 
to become an example of a statement by Gallagher (1964), "Nothing seems 
to direct the flow of research energies so much as a well-constructed 


measuring instrument". 
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III. RECOMMENDATIONS 


This study has introduced a method for measuring student per- 
formance during their actual interaction with a game model. The study 
provided information on the nature of the solution obtained by a compo- 
nent curve analysis of performance data for one group of students over 
one presentation of the simulation game. This study could represent the 
first step in a possible program of research on simulation games. 

Three recommendations for possible replication studies may be 
made. First, since the method relies on factor analytic techniques, the 
sample size should be increased. Second, if time permits, the number of 
cycles of the game should be increased to 20 or 30 trials. This would 
permit the study of component curves of greater length. Third, new vari- 
ables should be considered for inclusion into the set of external measures. 
The importance of the impulsivity measure in the present study illustrated 
the value of including variables from outside the cognitive domain. For 
example, new technological developments may soon permit the use of eye- 
movement and pupil diameter data that will indicate what aspects of the 
feedback information the student is attending to while using a simulation 
game. 

Another possible area of investigation would be the construc- 
tion and testing of models of student decision-making behavior. The pre- 
sent study examined one such model, that of random performance. More 
complex models could contain well defined strategies, which in turn could 
be related to a set of hypothesized abilities or personality measures, 
The interaction of such decision making models with the simulation game 


model would generate a set of performance scores that could be analyzed 
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by means of component curve analysis. The suitability of such decision- 
making models would be determined by comparing the results from the simu- 
lated performance with the results from an analysis of actual student per- 
formance on the game, 

The possibility of using component curve analysis to classify 
various simulation game models should be investigated in greater detail. 
The number and shape of component curves resulting from an analysis of 
random decision strategies may provide an index of complexity. 

The main recommendation is related to a programmatic approach 
to research on simulation oe Principles of experimental design should 
be applied to studies involving repeated playings of the same game. Also 
the structural characteristics of the game model should be systematically 
varied and the resulting effect on performance should be studied. The 
method of component curve analysis represents one method that could be 


used to measure and evaluate the results of these studies. 



















est ‘ aa ; at 
os 

-notetosb) rove 40 yatleédedive off) malentenn avgue Inemoqnes: 30 a 

-amie sia’ asd & ii feed afta tos Peqnes “ob b ec.) erat oy ad biaeiw ee ite 


sou titebut2 Isueas do Bieyvinn® ae herd eeliesr Sg dag sonwuso¥veg babel | 


<mg aels bad somnerag 
viizsals oj plavlsas ‘Swilo ansneenes afiee to vis hhdipeeg, ef | 


sh 1vatneto nt botnet! asvnt 3d biveda) siabom mg soidaluats 
A 
ulavinunk of mor yclJlVasst eels 3 re Oqmags ~36. agate boas zredmaun ad 


iomes 16 4 ‘1 (6 Shivoud vam 2elgpteade noketoeb 


dour if ; aro j of badefst 6? nobtisinbenages ‘nfom ent 
bluatts aoh 4 (Lasgx4 to zal pial ie ty noe ste Lute ng domeae 
7 


‘i? to/santve ly betaegay gatve Love vilouza oF batle 


fof 


v ant 3) ey | | far Py r ‘ ime sel A;0 oF ant : . : 
yi} . blugt® labor “aning oft to totdaiaedamtedo istujourt@ 1 


t- .bsi | bivoge soteaieiihe’ ae 229930 vitotoser edd bas 


-* 


. | 4 
ed Diuae 4) bole, ono ci gpSeaiQey Reaves: yt In anogmosS te IOs 





io etiveod off S¥Yaulayve baa -etvasse oo 





Beck, ia. and Monroe, & 


oe. Blishen, 


Bughaeli, 


SELECTED KEFEREWCES 


ApGerson, C, &, an Seperiment om bebavierg) Leare\og in « con 


fame. civulation 6 Games, i737", 


—— 


Sone dimension 


Taghwology. 3902,,9ct., 45 -. 4 


* 
B. AR. - eh Oe Oe Manic isace: i 
—_ i , - 
ad £ i 2 . T 
<7 : a 
i 
+t ‘3 


e 1 siers Ls 
mh x7 ars 8 — be AS ‘ 4 
SS, 6S, Af mial Stu 
wmaes wit leaciated Ex : 
at 1O6¢ 5 
r . eo 
an 
9 iin 
L#arriLés 'O.oO2% of eahir 
pputer-e Let 
‘6H 





2RaRs 4 


J’. and Sages 3 Y The unor 
"> — |) SELECTED ‘REFERENCES 


oad /. Cormnone it curve Ww 
computer-Sasea fimularic- 


Research, 197i, 2, 


Bs Ove &@ Alton. DB. Ww PAe 
Sew .feotk: John ; 


Car!son, E Learn { CniTrovsy ga 


= Ge, 964. 


Chapmar, &. L.. fannedy, EF, Le, Newe 
' 





fesedrin bavoratory’s air del«n 
Setenee, £959, 5, 250 =, 269 


D> -Cherryite ines, C. ,- murrent research of O88 
a yiau lactose oe pigestions for altarhs 


> 


6 = 
t 


» Riel, ¥. G, 
vaperi@ent se 








STIS GR: - 


125 


SELECTED REFERENCES 


Anderson, C. R. An experiment on behavioral learning in a consumer credit 
game. Simulation & Games, 1970, 1, 43 - 54. 


Beck, I. H. and Monroe, B. Some dimensions of simulation. Educational 


Technology, 1969, Oct., 45 - 49. 


Blishen, B. R. A socio-economic index for occupations in Canada. 


Canadian Review of Sociology and Anthropology, 1967, 4, 1, 
4l ye YSIS 


Boguslaw, R. The new utopians: A study of system design and social 
change. Englewood Cliffs, New Jersy: Prentice-Hall, 1965. 


Boocock, S. S. An Experimental Study of the Learning Effects of Two 
Games with Simulated Environments. American Behavioral Scientist, 
Ost. 1966 .- 8) lbs 


Boocock, S. S., & Coleman, J. S. Games with simulated environments in 


learning. Sociology of Education, 1966, 39, 215 - 236. 


Bundy, R. F. Computer-assisted instruction - where are we? Phi Delta 
Kappan, 1968, 49, 424 - 429. 


Burke, P. J. and Sage, D. D. The unorthodox use of a simulation instru- 
Meng. waoimilation «Games. 41970, 2, 155-171. 


Burnett, J. D. Component curve analysis of student performance on a 
computer-based simulation game. Alberta Journal of Educational 
Research, 1971, 2. 117 — 128. 


Bushnell, D. D., & Allen, D. W. (Eds.) The computer in American education. 
New York: John Wiley & Sons, 1967. 


Carlson, E. Learning through games. Washington, D. C.: Public Affairs 
Press, 1969. ; 


Chapman, R. L., Kennedy, R. L., Newell, A., & Biel, W. C. The system's 
research laboratory's air defense experiments. Management 
Science, 1959. 5, 250 >. 269. 


Cherryholmes, C. H. Some current research on effectiveness of educational 
simulations: Implications for alternative strategies. The 
American Behavioral Scientist, 1966, Oct., 4 - 7. 


Clarke, W. A research note on simulation in the social studies. 
Simulation & Games, 1970, 2, 203 - 210. 


Cohen, K..J.; iCyerG, Raia Di ll, We Rs, Juehn Aa A, . Miiner 6M... , 
VanWormer, T. A., & Winters, P. R. The Carnegie Tech management 
game. In H. Guetzkow (Ed.), Simulation in social science: 
Readings. Englewood Cliffs, New Jersey: Prentice-Hall, 1962, 




























tall 
v3 
~ 


enone NaH Od TRIAS, ne ie 


¥ on os 2 siavnel ied Ss cypnilou An Pract Sage! A an poner 
So 5 =) ; j mas a 2aaan a ie. Jyot® ., ig 


itenlunte to anorenahib sino aa , socinalt fm a ~ 4 aoe 
eé - ch ,. 350 . 208! xqosomets at 


- 


sf}: atotiaguoso 709 «xsherly a heonedesodese A 74 ad pee 
Oe reload fae yo) 650k 20 We ie feaare 
* oe oo ” > — —_ ‘ 7 
sf y 7 


A as 
l . neteve. to voRde «See Kanal On aif Al .Walwng 


. 


‘a prot. wok , e095 BF acme gel -BRfisds. 
‘ = : } nT 


a 
ve 


ati 455 uo] sd% Reo ehel? Leniemirege gAl 18.8 a 
iaitiwa yi tan : A «3 On y I vOa ms Satume? dtiw 2 4itts) 27 ata 
. ow - »@f - 8.8027 .350. 


* > OT 


-_ 
slimie H¢iw aengO 2 .l gratified @ 43d 8 4308 
GGL »terieube ye ee hey Led .gtisrieant; f 
q | Fredy, = ross ov itor by tetas sejuquaD ~ 7. or 


He Dae 9 Govt. faced 


lumie » to sev zebefiagey ed? 2 4 oegee Baa st tis 
Itl°- a N2t , eels € , hobialemte soon 
* 2 ge Aree em a i ll et 1 i 


eka j LA Th. Sy salQ Jtenragmod a s% 33 7 
_« NED. NRE S BL je la ‘baéad~tsJoqmon oe 
:A0t —~ Sil ,& .1VOt ~hosasesk 


4% OS Gs ; omega eit i aha) WW at} naila 3 yl a ate : 


; fase 8 yell edol. sare? walt ‘ 
BTIAItA oittud. +.3 .d .nedpeame aie -f9mey cguO oxd4 | ritniss.t * i o> 
: ak COOL ocak. Naa 


Wasitve off OW loth a ok ai town geal ft  gboaaed Jodie 
PRSh dy Ate M, giants sty wa ea 1¥ 96 Tt @* 7 aredsl donasee2 


j ; . ; « at - eS - « eC gRRE ae pre : | 


Suavidosiio yp Tore Minch ook, AH Ds | 
che a ee aaineanl 





126 


SELECTED REFERENCES 


Cooley, W. W. and Lohnes, P. R. Multivariate procedures for the behavioral 
sciences. New York: John Wiley & Sons, 1962. 


Darlington, R. B. Multiple regression in psychological research and prac- 
tice. Psychological Bulletin, 1968, 3, 161 - 182. 


Dawson, R. E. Simulation in the social sciences. In H. Guetzkow tgs be be 


Simulation in social science: Readings. Englewood Cliffs, New 
Jersey: Prentice-Hall, 1962. 


Dill, W. R., & Doppelt, N. The acquisition of experience in a complex 
management game. Simulation Models for Education, Fourth Annual 


Phi Delta Kappa Symposium on Educational Research, 1965, 71 - 104. 


Draper, N. R. and Smith, H. Applied regression analysis. New York: John 
Wiley & Sons, 1967. 


Duncanson, J. P. Learning and measured abilities. Journal of Educational 
Psychology, 1966, 57, 220 - 229. 


Eckart, C. and Young, G. The approximation of one matrix by another of 
lower rank. Psychometrika, 1936, 1, 211 - 218. 


Edwards, A. L., & Cronbach, L. J. Experimental design for research in 


psychotherapy. Journal of Clinical Psychology, 1952, 8, 51 - 59. 
Falkoff, A. D. and Iverson, K. E. APL\360: User's Manual. Thomas J.Watson 


Research Center: International Business Machines Corporation, 
1968. 


French, J. W., Ekstrom, R. B., & Price, L. A. Manual for Kit of Reference 
Tests for Cognitive Factors (Revised 1963). Princeton, New 
Jersey: Educational Testing Service, September 1967 Edition. 


Gallagher, J. J. Meaningful learning and retention: Intrapersonal cog- 
nitive variables, Review of Educational Research, 1964, 5, 
499 7 5125 


Hakstian, A. R. and Boyd, W. M. On the general "orthomax" criterion for 
orthogonal factor transformation. Technical Report No. 10, 
School of Education, University of Massechusetts, 1970. 


Harman, H. H. Modern factor analysis. (2nd ed. revised) Chicago: The 
University of Chicago Press, 1967. 


Harris, C. W. On factors and factor scores. Psychometrika, 1967, 4, 
363 - 379. 


Helburn, N. New materials and teaching strategies: The High School 
Geography Project, The Bulletin of the National Association of 


Secondary-School Principals, 1967, 50, 20 - 29. 



























2AM SS GATOS IR 7% 
s4* tot-eetubeoosg Saeiskvio fam’ "ao 7a Bee” Ww ow wero | 


nal _ u et a eae 
eo eT 273 Ae 2 ~ WwW nilok eb oY, ey .* 


Hee) Lestkoledoyeq af toh AwAROL stern a | moan teat 


Pe 1; RHGT he Li Pi ni> .a5ts 7 
; oft ks a. ra | aoe ¥ 
isise IaTack $ri3 nt aultefoay? ae | ne 1 
alipat*pasqerqe lasyee Sl aOsSeR ears 
ie. tia “asi Inet8 ryese Tol % 
y ok “prev iz m4 yn) WAT iM , Shaqgot é el ‘ f 
Sabot aGriptsmt?  . cig toameganae : 
meu i an Lie vce? stint moto Fae 
j : 7 i 7 ~~ 
| roy, ittebter Eettgat i ata bia > Bch ieee 
: ot ,ano®? & ypliW- ~ Case 
=) Pad Ps Bean Se, saitrioeTt ig 8 , TORE 
% ooo TG «ve cased tao lodge? 
Ro mobs ‘xox gym ont '« 2 ,gnvoY bra wD (278s 
‘$i; — eis *! pt, t : ips NE: aay betas " a 
, ~ ‘ ~~ 4 -. : — 
ii 6 A LM oh DS Di) | Boe 7 «kL es ,foadi@® F-} cee. «A ¢ owed. 
<n if =% tasloi ty tego eqatadratoysgq 
r , 
n ai4ieA ot ot heel bag 5A Soe 


ua Ineattengered : seated Hotasepd — 
és .Babj 


A, tk SS ee alah ac ae err 


wo hag rs 4 noe yt Pea ha tage toi a3asT ~ 1 
rf irve? oagiae! loeotdaoyiZ- Jey 


“BED 1MG8 te idistS? par, antetaol iaiypetaeen 2 Pa \setgata 
iat) MOL: | ISP CE “unoky hwitad = wo. ined anldhyney evisin e 
bat hi “Sie 4 ta Tk ie = 











'» 





gmedt oo!" Lp ppg wate ao: 6M -byon 
$e: = es es | re Pa 
tela itt 






, el “ed i399 
Of. dOR dotge! saocaieol’ sate 
Bh aly oy eae Se 


aa 


yr yt +8 i. 
Alene es 


ee 
{Se 


127 


SELECTED REFERENCES 


Helburn, N. The educational objectives of high school geography, The 
Journal of Geography, 1968, 5, 274 - 281. 


Hendren, P. Computer-aided design in architecture. Brief Report of the 
Fifth ONR Conference on CAl, January 30 - 31, 1968, Newburyport, 
Mass.: Entelek, Inc. 


International Business Machines Corporation, System /360, Scientific 
Subroutine Package (360A-CM-03X) Version III, Programmer's 
Manual, Ref. No. H20-0205-3, p. 77. 


Jacobson, C. A. The application of the hybrid computer in flight simula- 


tion. Proceedings of the IBM Scientific Computing Symposium on 
Computer-Aided Experimentation, 1966, 206 - 244. 


Joreskog, K. G. Some contributions to maximum likelihood factor analysis. 


Psychometrika, 1967, 4, 443 - 482. 


Kagan, J., Pearson, L., & Welch, L. Conceptual impulsivity and inductive 
reasoning. Child Development, 1966a, 37, 583 - 594. 


Kagan, J., Pearson, L., & Welch, L. Modifiability of an impulsive tempo. 
Journal of Educational Psychology, 1966b, 57, 359 - 365. 


Kersh, B. Y. Classroom simulation, a new dimension in teacher education. 
1963 Training Research Division, Oregon State System of Higher 
Education Title VII NDEA Project 886, Monmouth. 


Koch, E. D. Gaming, play, and education. Journal of Educational Thought, 
HIS; §275078.- 90. 


Kurfman, D. Improving the new georaphy through evaluation. The Bulletin 
of the National Association of Secondary-School Principals, 1967, 
50, 37-243. 


Lagowski, J. J. Simulation in chemistry. In Brief Report of the Fifth 
ONR Conference on CAl, January 30-31, 1968, Newburyport, Mass.: 
Entelek, Inc. 


Lekan, H. A. (Ed.) Index to computer assisted instruction. Milwaukee: 
Instructional Media Laboratory, University of Wisconson, 1969. 


Lekan, H. A. (Ed.) Index to computer assisted instruction, Second edition. 
Milwaukee: Instructional Media Laboratory, University of Wisconson, 
19702 


Lemke, E. A., Klausmeir, H. J., and Harris, C. W. Relationship of selected 
cognitive abilities to concept attainment and information process- 


ing. Journal of Educational Psychology, 1967, 57, 27 - 35. 


i an . ae 


Vy 
so digit losmeytacetdo Ineokiaaghe ae cot 


fag + Skee Oe! aie Sey $y: 




































© 


sartaitaa qyTOMaAE _ i 


ef? ,yiqezgzesn 100 


* 


yviosl itty et agtest babi tesa? Ji) neabe 


Sri { ‘ i .~ a 
+o 25 it ~ Of gafurel [AD a0 4o0stanee. A eee 
soni , tol adel 7. e3eh 
53235 WE’ meveve . pons sc aoninoal zeadioitt taco tianza 


a yonmp7Q074 , Lit wotate¥) CMO Nes: ) Saale ee eeeeeee: 7 os 

| Te be CES sR at, tet , tsunsh ‘ 
, poltaatleqaqent LA sa 
st ij< MET ie j saothgegonry rit) oe : 
7), | akie sats jo SehebA- secemBd shee 


at boonrleNil wmegreaw ef Sree Jeg aoe «2 whe 


«$3 Pp. (@ Taal parsseno moioged 
hoi i‘ §] Bead pehsnet ea : 
1 E) ,EOIOI , InSeaol bese! DIES, .gaeeoe Res 


t F nu j 4 of | as ; NG ft Ho a Ps) ® et > ItGe 79847 eeu ef 
A £7? , desetoeentonovet Demyeanag i ae iaazue’ Sal 


rit i 19 aadee8o? ab aegenalk che Cons ‘ ols Jip! ihe, dabiaats ¥ .@ 2h 
E SIAC 10  ¢ceaawigd toootedd gitetaxT COW 

fjgomnolt .gBG shotSeS S04 Di eee ae isagyee. eee 

: rs a ; .) ‘e 7 

WT tandotaoor . wiuoh ,odsecu8s boeocele egirimad. ud a sae 

' oe ’ 

ue = By a _, det 


— 


itaal Li Pe ~ eer} s tpuet 4709 FOG SOSE, tlt gh 3 gaivorqml aa «ft 
,Wel ,aleqis . ‘2 >n0ode. 26 Bi etsoted tggeeet vem2 Re 
| «te ~' XE hi 

pe ore : 

BISi3. i) Jy siyqet Luis oF, qapeammaionas senate a ak , daewi 
bent ,Iogytidwons B00), UE OG “amass IAD a6 sortgaatagd ; 


‘a | | oe ,Hol are 
earn Anemetedennd> 03 ieabe ue mT ce 


Sua NGatEaines 605 tanebacarszegt 
7 byengd 


-. 






















128 


SELECTED REFERENCES 


Levinthal, C. Computer construction and display of molecular models. 


Proceedings of the IBM Scientific Computing Symposium on 
Computer-Aided Experimentation, 1966, 315 - 326. 


Moss, J. H. Commentary on Harling's simulation techniques in operations 
research. Operations Research, 1958, 6, 591 - 593. 


Naylor, 0. Hw, Balintiys J.Pi° Burdick, DIS, oe Kong, C. Computer Simu- 
lation Techniques. New York: John Wiley & Sons, Inc., 1966. 


Raser, J. R. Simulation and society: An exploration of scientific gaming. 
Boston: Allyn and Bacon, Inc., 1969. 


Reed, S. C. Some relationships between conceptual complexity and mental 
abilities. Research Bulletin 66 - 33, Educational Testing 
Service, 1966. 


Ross, J. Mean performance and the factor analysis of learning data. 


Psychometrika, 1964, 29, 1, 67 - 73. 


Samuelson, P. A. Foundations of economic analysis. Cambridge: Harvard 
Univ. Press, 1947. 


Stahl, A. F. Mode of presentation and subjects! affective reactions to 
the resolution of simulated problems. Simulation & Games, 1970, 
B57 .203 7-3 200. 


Stolurow, L. M. Essential principles of programmed instruction. Technical 
Report No. 8, June, 1965, Training Research Laboratory, University 
of Illinois. 


Tansey, P. J. Simulation Techniques in the Training of Teachers. 
Simulation & Games, 1970, 3, 281 - 303. 


Tucker, L. R. Factor analysis of double centered score matrices. Princeton, 
New Jersey: Educational Testing Service Research Memorandum 
RM-56-3, October 1956. 


Tucker, L. R. Determination of generalized learning curves by factor ana- 
LySis.9)Princeton: ~Princeton Univ. and\Educe Test. ssery loos 


Tucker, L. R. Learning theory and multivariate experiment: Illustration 
by determination of generalized learning curves. In R. B. 


Cattell (Ed.), Handbook of multivariate experimental psychology. 
Chicago: Rand McNally & Co., 1966. 


Verba, S. Simulation, reality, and theory in international relations. 
World Politics, 1964, XVI (3), 491. 


Wampler, J. F. Prediction of achievement in college mathematics. 
Mathematics Teacher, 1966, 59, 364 - 369. 










Ae ATS TANI MS 


etebow tsluostom to gates? bak narseysamee TRIOS: - ,isdanived — 
; > 
eo mo teomme® gent Tineath). cbt eiNRSs EE Sie a 


: ae? ; | AOD 4 Aids hee IS A. 3abhh- 1830qm09 
iS 


doa we  eteteanat at -L ,380M | 
roltosegO .fevesent 7 


gimmie e* aor 


ar ; 
} 
f . [04 (8: Bee Aeseesen 2 
- _ » ——_ - 





ey ) ,gwok. 3. . 8) pi aria tok gtpatlad a Tb tol galt 
‘ye nAol “sao G AeA ,isveindent mo kisi an 
Ps 



























ior nA ted et oR ee nok sw fant2 oh «6 @8 


“oa8! , Shr anced Lan aie taqg?ach + 
{qoones gas ton! an hedot ta iew woe. .3 sd € 

, vA ctaatiodt dovpoesh ..eetstiids vs 
200! ,eoivise 558 

uta 10198? of% bom eetthemolreg agent 

-y « A ' _ oot ene oaeraieens 


0 ri attot rig 7 | eaene 
ae | .waard «vind Y 
a ese 33 wt Sia Unie Testes aagd So sho 9 WAU 


icfimele 35 eekfulorer any 
/ .0S¢ ™ ras m4 


ln ) . , 0 teintgatiq [skimeren Mad ? 
Layo “rd gacad ‘istext ,@0C) enh ef .of croqet 
i «#toatiit toe 


orf edt el gaen titaaT roetatumie® .L 2 xan 
1 - (MG Eg Orel 4 , 2ome S-noliel wie 





~AIsoqitd Beotuten stooe | toss, stduoh te ateciem wIset 3 «A ed g ISR 
RebhsioneM Poses2o% ap ivibhe \aetaged henck agape gysetel wall | ae 
300) uadgaad ge-de=m 










1B Suton! qd aavine acictnel Lestivesnng Jo cotsepieeted 6A ‘ail 47089 
m- , pao dent obit ban ae, ia a -atagl i a 










i ae 


it iol ne ik aa 


129 


SELECTED REFERENCES 


Watson, C. This geography is something to sing about, American Education, 
1969, Oct. 


Weitzman, R. A. A factor analytic method for investigating differences 
between groups of individual learning curves. Psychometrika, 
1963, 28, 1, (69 =200- 


Winer, B. J. Statistical principles in experimental design. New York: 
McGraw-Hill, 1962. 


Wing, R. L. The production and evaluation of three computer-based econo- 
mics games for the sixth grade. Cooperative Research Project 
2841, June, 1967, The Board of Cooperative Educational Services, 
First Supervisory District, Westchester County, New York. 


Zuckerman, D. and Horn, R. (Eds.) Guide to simulation games for education 
and training. Cambridge, Mass.: Information Resources, Inc., 


1970. 












. we 
aoanavan “deme ee 
anijaou™!. pesrvacs , v0e ante of ghiesonge a} eigeigoey arith 2 am 
— “+290 «8822 


hae ol - i HVe Deak on > ty iarth r0oe% A | oA i ataorstt hdl 
p'J ; j sar Tho oP er ipplrvidnt Ro 2Qeo1g vaavigd a 
. = % ad «St bee! at ae 


+ 


‘ at ! 5 sl +f «> hh 
wl « 1 tii-waidoM : a 


* 


» sawld So nope base no! Tauber odT «J. A 
iwiieseo a) \abhee Meike ait set Semag eslm 
ub ovies7e0e6D Jo-btaok of ,FORL geal ees 
snuJD 1ssascoheeW pss tite ith ypee Rureque-sealis 


— 


ja-o? Sbiud tvaie? 3 ,w208 bap -0-,asmt at yas! 


tai 7 FPO .osbs ican ‘Saiatet3 bus ~~ 
; OTOL 








i 


< Dor 2 


Se . 
APPENDICES 


SAMPLE BALES € 


REGRENCE TI 


rs 





APPENDIX A 


SAMPLE PAGES FROM THE 


KIT OF REFERENCE TESTS FOR COGNITIVE FACTORS 


HOTS EIDAT eee 






pK WIR B54 


) Aa DIPS. ROMMMNNRA YO TIN 37 


132 
Name: 





LETTER SETS TEST — I-1 


Each problem in this test has five groups of letters with four letters 
in each group. Four of the groups of letters are alike in some way. You 
are to find the rule that makes these four groups alike. The fifth group 
is different from them and will not fit this rule. Draw an X through the 
group of letters that is different. 


NOTE: The rules will not be based on the sounds of groups of 


letters ; the shares of letters, or whether letter combinations 
form words or parts of words. 


Examples: 


A. NOPQ Dat ABCD HIJK UVWX 


B. NLIK PLIK QLIK Tek = VLIK 


In example A, four of the groups have letters in alphabetical order. 
An X has therefore been drawn through DEFL. In example B, four of the 
groups contain the letter L. Therefore, an X has been drawn through THIK. 


Your score on this test will be the number of problems marked correctly 
minus a fraction of the number marked incorrectly. Therefore, it will 


not be to your advantage to guess unless you are able to eliminate one or 
more of the letter groups. 


You will be allowed { minutes !'or each of the two parts of this test. 
Fach part has 1 page. When you have finished Part 1, STOP. Please do not 
go on to Part 2 until you are asked to do so. 


DO NOT TURN THIS PAGE UNTIL ASKED TO DO SO. 


Copyright ©) 1962 by Educational Testing Service. All rights reserved. 
Suggested by Letter Grouping by L. L. Thurstorie 
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FIGURE CLASSIFICATION — I-34 


This is a test of your ability to discover rules that explain things. In each 
problem on this test there are either two or three groups, each consisting of three 
figures. You are to look for something that is the same about the three figures in 
any one group and for things that make the groups different from one another. 


Now look at the sample problem below. In the first line, the figures are 
divided into Group 1 and Group 2. The squares in Group 1 are shaded and the squares 
in Group 2 are not shaded. In the second line a 1 has been written under each 
figure that has a shaded square as in Group 1. A 2 has been written under each figure 
with an unshaded square as in Group 2. 


Group 1 Group 2 






Group 2 


The figures in Group 1 consist of both straight and curved lines. The figures 
in Group 2 consist of curved lines only. The figures in Group 3 consist of straight 
lines only. As you can see, there are other details that have nothing to do with the 
ruie.. The auswers: are: 1, 1, 5,11, 2, 1, 2; 2. 


Your score on this test will be the number of figures identified correctly minus 
a fraction of the number marked incorrectly. Therefore, it will not be to your ad- 
vantage to guess unless you have some idea of the group to which the figure belongs. 


- You will have 8 minutes for each of the two parts of this test. Each part has 4 
pages. When you have finished Part 1, STOP. Please do not go on to Part 2 until you 
are asked to do so. 


DO NOT TURN THIS PAGE UNTIL ASKED TO DO SO. 


Copyright 1962 Educational Testing Service. All rights reserved. 
Adapted from a University of North Carolina test. 
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OBJECT - NUMBER TEST —— Mg-2 


This is a test of your ability to learn combinations of words and 
numbers. In each part of the test you will study a page showing 15 
object names with numbers. After studying the page showing both 
objects and numbers you will turn to a page showing the nemes of the 
objects in a different order. You will be asked to write down the 
numbers that go with them. 


Here is a practice list. Study it until you are asked to turn 
to the practice test page (1 minute). 


window 16. 
desk Ta 
carpet 19 
door 84 
glass 90 


DO NOT TURN THIS PAGE UNTIL ASKED TO DO SO. 


Copyright (©) 1962 by Educational Testing Service. All rights reserved. 
Adapted from Word-Number by L. L. Thurstone 
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FIRST AND LAST NAMES TEST — Ma-3 


This is a test of your ability to learn first and last names. 
In each part of the test you will study a page of 15 full names, 
first and last. After studying the page showing full names you will 
turn to a page showing a list of the last names in a different order. 
You will be asked to write the first names that go with each last 
name. 
Here are some practice names. Study them until you are asked 
to turn to the next page (1 minute). 
Janet Gregory 
Thomas Adams 
Roland Donaldson 


Patricia Fletcher 


Betty Bronson 


DO NOT TURN THIS PAGE UNTIL ASKED TO DO SO. 


Copyright 1962 by Educational Testing Service 
Adapted from First Names by L. L. Thurstone 
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ADDITION TEST — N-1 


This is a test to see how quickly and accurately you can add. It 
is not expected that you will finish all the problems in the time 
allowed. 


You are to write your answers in the boxes below the problems. 
Severe.l practice problems are given below with the first one correctly 
worked. Practice for speed on the others. This practice may help 
your score. 


Practice Problems: 


12 84 ii 3h 17 k5 31 80 
54 38 81 50 Tal 52 78 


7 
6 y) 
io a a ee 
Your score on this test will be the number of problems that are 
added correctly. Work as rapidly as you can without sacrificing accuracy. 


_— 
E"] HO £ 


You will have 2 minutes for each of the two parts of this test. 
Each part has one page. When you have finished Part 1, STOP. Please 
do not go on to Part 2 wntil you are asked to do so. 


DO NOT TURN THIS PAGE UNTIL ASKED TO DO SO. 


Copyright 1962 by Educational Testing Service 
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SUBTRACTION AND MULTIPLICATION TEST — N-?, 


This is a test to see how quickly and accurately you can subtract 
ani multiply. It is not expected that you will finish ail the problems 
in the time allowed. 


You are to write your answers in the boxes below the problems. 
Several practice problems are given below with the first one correctly 
worked. Practice for speed on the others. This practice may help 
your score. 


If you wish, you may use the space between the lines or at 
the bottom of the page for scratchwork. 


Practice Problems: 


Subtract: 

68 4O a7 ie 81 2 90 46 56 
= =p Sa = -38 = aoe sad =a) 
Gail mele tied He Sa Ee ae | 
Multiply: 

5 a a Re 86 s 69 


eat cea ea ate =e eeaeg 


Your score on this test will be the number of problems solved 
correctly. Work as rapidly as you can without sacrificing accuracy. 


You will have 2 minutes for each of the two parts of this Gest. 
Each part has one page. When you have finished Part 1, STOP. Please 
do not go on to Part 2 until you are asked to do so. 


DO NOT TURN THIS PAGE UNTIL ASKED TO DO SO. 


Copyright 1962 by Educational Testing Service 
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MATHEMATICS APTITUDE TEST — R-1 


In this test you will be asked to solve some problems in mathe- 
matics. Solve each problem and put an X through the number in front 
of the answer that you select. 


Example 


How many pencils can you buy for 50 cents at the rate of 
2 for 5 cents? 


1-10 
2-20 
2722 
4-100 
2-125 


Your score on this test will be the number marked correctly minus 
a fraction of the number marked incorrectly. Therefore, it will not be 
to your advantage to guess unless you are able to eliminate one or more 
of the answer choices as wrong. 


You will have 10 minutes for each of the two parts of this test. 
Each part has 3 pages. When you have finished Part 1, STOP. Please do 
not go on to Part 2 until you are asked to do so. 


DO NOT TURN THIS PAGE UNTIL ASKED TO DO SO. 


Copyright © 1962 by Educational Testing Service. All rights reserved. 
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NECESSARY ARITHMETIC OPERATIONS TEST — R-4 


This test consists of problems in mathematics. However, instead of 
solving the problems and finding an answer, your task will be merely to 
indicate which arithmetic operations could be used, if you solved the 


problems. Put an X through the number in front of the option that you 
select. 


Example I 


If a man earns ee an hour, how many hours should he work each 
day in order to make an average of $22.50 per day? 


1-subtract 
e-divide 
J-add 
4-multiply 


In order to solve the problem you should divide $22.50 by $2.75; 
therefore, you should have put an X through 2. 


Example IT 


Desks priced at $40 each are being sold in lots of 4 at 85% of 
the original price. How much would 4 desks cost? 


1-divide and add 
e-multiply and multiply 
4-subtract and divide 
h-multiply and divide 


One way to solve the problem would be to multiply $40 by .85 and 
then multiply this product by 4; therefore, you should have put an 
X through number 2. (Although some problems may be solved in more 
than one way, as with Example II, only the operations for one of these 
ways will be given among the options). 


When 2 operations are given, they are always given in the order in 
which they should be performed. 


Your score on this test will be the number marked correctly minus 
a fraction of the number marked incorrectly. Therefore, it will not be 
to your advantage to guess unless you are able to eliminate one or more 
of the answer choices as wrong. 


You will have 5 minutes for each of the 2 parts of this test. Each 


part has 3 pages. When you have finished Part 1, STOP. Please do not 
go on to Part 2 until you are asked to do so. 


DO NOT TURN THIS PAGE UNTIL ASKED TO DO SO. 


Suggested by Necessary Operations by J. P. Guilford 
Copyright ©) 1962 by Educational Testing Service. All rights reserved. 
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VOCABULARY TEST — V-1 


This is a test of your knowledge of word meanings. Lock at the 
sample below. One of the four numbered words has the same meaning or 
nearly the same meaning as the word at the left. Indicate your answer 


by writing, in the parentheses at the right, the number of the word 
that you select. 


attempt 1-run 2-hate 3-try LaELOp Mm Vk tone ye ae Leas) 


The answer to the item is number 3; you should have a "3" written 
in the parentheses. 


Your score will be the number marked correctly minus a fraction 
of the number marked incorrectly. Therefore, it will not be to your 
advantage to guess unless you are able to eliminate one or more of the 
answer choices as wrong. 


You will have 4 minutes for each of the two parts of this test. 


‘Each part has one page. When you have finished Part 1, STOP. Please 
do not go on to Part 2 until you are asked to do so. 


DO NOT TURN THIS PAGE UNTIL ASKED TO DO SO. 
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Adapted from a test by J. B. Carroll 
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VOCABULARY TEST — V-2 


This is a test of your knowledge of word meanings. Look at the 
sample below. One of the five numbered words has the same meaning or 
nearly the same meaning as the word above the numbered words. Mark 
your answer by putting an X through the number in front of the word 
that you select. 


jovial 


1-refreshing 
2-scare 
3-thickset 
4-wise 


Bcjolly 


The answer to the sample item is number 5; therefore, an X has 
been put through number 5. 


Your score will be the number marked correctly minus a fraction 
of the number marked incorrectly. Therefore, it will not be to your 
advantage to guess unless you are able to eliminate one or more of the 
answer choices as wrong. 


You will have 4 minutes for each of the two parts of this test. 
Each part has one page. When you have finished Part 1, STOP. Please 
do not go on to Part 2 until you are asked to do so. 


DO NOT TURN THIS PAGE UNTIL ASKED TO DO SO. 


Copyright © 1962 by Educational Testing Service. All rights reserved. 
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APPENDIX B 


DOCUMENTATION OF COMPUTER PROGRAM 


FOR SIMULATION GAME 
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COURSE DOCUMENTATION 


Course Title: ftshy 

Educational Level: Junior High School 
Author: J. Dale Burnett 
Programmer: J. Dale Burnett 
Computer: IBM 1500 System 
Language: Coursewriter Il 
Date: December, 1970 


Division of Educational Research 
The University of Alberta 


Edmonton, Canada 
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fishy 


Information for Educator 


ih 


ae 


Objectives and Purpose of Program 


The primary reason for writing the program was to collect data 
on student performance on a simulation game for use in a Ph.D. 
thesis. However the simulation of the fishing industry was 
selected because of its' relevance to the Junior High School 
Social Studies curriculum on primary industries. It is 
expected that this program could be incorporated into an 
existing unit on primary industries. 


Course Content 


The program gives a brief description of the importance of the — 
salmon fishing industry. This is followed by the actual simulation 
exercise. There are four distinct versions of the simulation 

game, referred to as Model 1, Model 2, Model 3, and Model 4. 

These models are described in section 6, Method of Instruction. 


Educational Level 


The content is suitable for Junior High School Social Studies 
curriculum. The program is suitable for all age levels over 12 
years. Since the program contains 4 models of increasing complexity, 
younger students may find the larger models too difficult, however 
at present there is no data available to make an evaluation of 

this possibility. 


Source of Course Content 


None. 


Program Duration 


No data available at time of documentation. An estimate of one 
hour appears reasonable. 
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6. Method of Instruction 


Simulation game 
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MODEL 1: 
a) Objectives 
i) the student should understand the relationship of 
market price and amount of fish caught to cash received 
ii) the student should understand the relationship between 
number of fish caught and resultant fish population 
for the following year. 
b) Form of equations 
initialization 
M, = 10 
Vp = 500 
Sr = 100 
Fo = 300 
equations 
eas —5e< < 
Sr Sr + R, Ae) R,< 5 (1) 
= M1 +R » “1s Rs 1 CZ) 
Fo = Se (Fy C.) /70 (3) 
Vp = Vp + (C,, MM, /10) (4) 
where S,_, represents reproduction and survival factor 


T 


Mr 


Fr 


Cy 


V 
T 


R}> 


represents 
represents 
represents 


represents 


the market price (a stochastic variable) 
number of fish at time T 
number of fish caught at time T 


value of cash on hand at time T 


R, are random numbers 


2 


Therefore model 1 contains 2 random variates, 5 variables, and 
The student has direct control over 1 variable, Co 


4 equations. 
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MODEL 2: 


a) Objectives 
i) objectives of model 1 


ii) the student should understand that the number of 
employees affects the number of fish caught. 


iii) the student should understand the relationship between 
value of equipment and both the cash on hand, and the number 


of fish caught. 


iv) the student should understand the value of a balance 
between number of employees and value of machinery. 


b) Form of equations 


initialization 
M, = 10 
Ve = 500 
Sy =" 7100 
Fe = 300 
Er = 400 
Pr = 150 

equations 
Se = Spey + Ry oy ee =5 Ce 

= ake as Ro » -1 < al (2) 
Ere See pat NS (3) 
Pp Pp + Ap 4) 
“Tm FED Py) ©) 
Fo = Sa (Foy - C)/70 (6) 
Mie oa Oh Re ACE AKY ees 7 (7) 
where ES represents number of employees at time T 

No represents number of new employees at time T (may be 0) 
Pr represents value of fishery 
Ay represents value of an addition or maintenance to 


plant or equipment 
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is given in graphical form at the end of this section. 


The corresponding analytic description is as follows: 
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Spee Sos 
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b)* 4£824075< 
PAT) Op ge ey ee 
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5] 


420 


> 420 


then 
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c) 


Therefore model 2 contains 2 random variates, 9 variables, and 


7 equations. 
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MODEL 3: 


350 , ='then 
Af, BOs Py < 240 
£f 2407s Pr < 420 
it Po > 420 


fishy 
36 10 
Rep res oe a 
100 10 
ter Ui Fy 
100 10 
6.24 Py 420 | Fy 
267 10 


148 


The student has direct control over 2 variables, No and 


a) Objectives 


i) 
1S. 


iii) 


iv) 


v) 


b) Form 


objectives of model 2. 


student should understand concept of depreciation 
and the need for maintenance of equipment. 


student should be aware of a natural turnover Ovescarn. 


student should be aware that payment of salaries must be 


taken into consideration when determining number of 


employees. 


student should be aware of demands for raises. 


of equations 


initialize 
Ve = 
Sp 

Fy = 


tion 
10 
500 
100 
300 


400 
150 
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6 

equations 
a eee a car eee (1) 
M, = Mp es », -l< Ry a (2) 
By) =e Be X, - RAED osRk, 5 0.09 (3) 
Pe aed ve = Oat ey (4) 
ire 2 (E, Pp) (3) 
if Toe 22 3 ObEO 5 Hy mee Hoy (6) 
ers, (Ge Gp) / 70 (7) 
Vn = Wey tee, My) / 10 -A, - Hp E, /200 (8) 


where R, is a random number representing rate Of men leaving their job. 


H,. represents the average salary at time T. Therefore model 3 


contains 3 random variates, 10 variables, and 8 equations. The student 
has direct control over 2 variables, Ne and Ape 


MODEL 4: 
a) Objectives 
i) objectives of model 3 
ii) student should understand that water pollution will 
have a detrimental effect on the survivial factor of 


the fish. 


4ii) student should realize the benefits that may result 
from construction of fish ladders. 


b) Form of equations 


initialization 
M,. 10 
Vp = 500 
Sp 
Fr 


= 100 
= 300 
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htshy 150 


ys 

= 400 

= 150 
Hp = 30 
bea LOO 

3 
Dy = 150 
equations 
ee” hee aoe (1) 
Dpi = Perey rats (2) 
Srlis horse teen ti Ore (L,/18 + D,/21) $72 sR, = 5 (3) 
Men i My ORs “w-ls Rost (4) 
ee eae eave eee ga , 0 <R, < 0,09(5) 
Bombay gar fe Ost ery (6) 
if t= 43 or7a 7. H, = We) A, (7) 
Cy = f (Ey P.) (8) 
Fee pS C,.)/70 (9) 
Vie = Vp_y + (CM) LALO. -= Ap = He E,./ 200 = X = Wr (10) 
where Ly represents value of fish ladders at time - 

Xo. represents amount spent on ladders at time T 
De represents value of purification equipment at time T 


W represents amount spent on purification at time T 


Therefore model 4 contains 3 random variates, 14 variables, and 10 
equations. The student has direct control over 4 variables, Xo Woe Noo and Ane 
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Audio-Visual Requirements 


None. 


Response Modes 


Keyboard and light-pen. 


Student Operating Requirements 


None. 


Teacher Supervision 


Not required. 


Evaluative Information 
Standard performance recordings are maintained. No data has 


been analyzed at this time. This program will be used in the 
programmer's thesis. 


Auxiliary Materials 


None. 


Student Reaction 


Not available at time of documentation. 


Programmer and Author 


ee Da lembumiete 


Availability 


Unrestricted, but permission should be requested of author. 
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Information for Operator 


rs 


a 


Course Name and Segments 
bishy-000 


Dictionaries and Graphic Sets 


Standard dictionary only. 


Functions Called 


mv, ek, Aa. 


Macros Called 


duadd7, tkago7. 


Film Reels 


None. 


Audio Tapes 


None. 


Execution Time 


Approximately 1 hour. 


Response Time 


_ Response times are included with ep instructions. 


Pre-Course Instruction Requirements 


The "student name" field at the time the student is registered must conte 
a one-digit integer (1, 2, 3, or 4) to indicate the simulation model. 

the student is to use. A 1 indicates Model 1 a 2 indicates Model 2, 

etc. A description of the models is given in the section, "Information 
for the Education, 6. Method of Instruction." The operator should 

ensure that this information is provided by the person requesting the 
program. 
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10. Student Sign-On Command 


Standard sign-on. 


ll. Teacher Supervision 


Not required. 


12. Proctor Messages 


Message: INCORRECT REGISTRATION 
ENTER THE MODEL NUMBER (1, 2, 3, 4) 
AT THE STUDENT STATION 


Reason: The program extracts an integer from the "student name" 
field in buffer 0 after the student signs-on. The 
above proctor message will appear if a value of 1, 2, 3, 
or 4 is not encountered. 


Action: The following message will appear at the student terminal- 


PLEASE WAIT FOR THE PROCTOR TO COME TO YOUR 
TERMINAL. 


The program will be at an ep instruction. ‘The proctor 

must enter a one digit number (1, 2, 3 or 4) at the student term: 
This value corresponds to the model of the fishing industry that 
the student is to use. This information should have 

been supplied at the time the user requested the scheduling 

of the program. A further description of the four models 

is given in the section, "Information for the Educator, 

6. Method of Instruction." 


Message: Please help this student use the l-pen. 


Reason: The student is required to point the light-pen at the 
"target' after the word END. The student has timed out 
after 100 seconds. 


Action: (1) Explain the use of the light-pen to the student. 
(2) Have the student point to the 'target', the course 
will then continue. 
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Message: Help this student use the keyboard. 
Reason: The student is required to type the number 1960. The 
student has either timed out after 2 minutes or a 
value other than 1960 has been entered. 

Action: (1) Explain the use of the keyboard to the student. In 
particular explain the use of the integers on the 
top row of the keyboard (including the aumbers 1 and 
0). 


(2) Explain how to enter a response. 


(3) Have the student type the number 1960, the course 
will then continue. 


13. Performance Recordings 


Performance recordings will usually be made. 


1a Special Instructions for Operator 


See 9. Pre-Course Instruction Requirements and 12. Proctor Messages. 


; tel sah.) = - 














ci 


. 


Winod tet af? Sen theboss Blis gieh 


WT O01 ssdintn sd 2g oF bapigast>s) sneha eat 
ro =950Ntm & tortn JU6 bbaEy tenses capil Jnwhitts 
: oyietns 50d eba ager pels tots. sntlev 7 


ni .tnetwax sigsos Brood sdy Yo ey de mba tee Sn a 


ae | : ? ; ; ij as hsinxs retos? mag 
i i af7. une 7) hysody~an vi to Woy qo? ; 


ROgGSst & Yesneoes wor ote lek (S$: . 


‘ 7 


esr S agg 3 ag huse) aus "oR *(E) 
suetsnes natr Tipe 


log 4. « j : 
tw Ntoost Aameeieet~ Cf 
~ [hie Lijw é:9 eae Soleeetanet 
‘Jatt 72) ego sie (elsegS ee 
ee ee i fas otzsvt ead setaedeert (ff eet 
i 


* 





fishy 156 


Information for Programmer 


1. Course Listing and Flow-Chart 


360 output of Documentation is enclosed. 360 output of How- 
Charting program is enclosed. 


2. Macros Called 


dua007 - press space bar to continue. 


4La007 - time out. 


9. Special Programming Features 


The value of buffer 0 after the student signs-on is searched using 
function et (Extract integer) and the numerical value is stored in 
counter 29. This value must be 1, 2, 3, or 4. The logic branches 
to the appropriate simulation model (1, 2, 3, or 4) depending on 
the value of counter 29. 


This procedure makes use of the fact that the value of the "student 


name" field at the time the student number is registered is stored 
in buffer 0 when the sign-on procedure is completed. 


10. Programmers Name and Date of Documentation 


Je Dalle Burnece. 
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4. Counters Used 


Since counters are used extensively in this course, the following 
table is provided to give the correspondence between the variables 
in the game models and the counters used in the program. 



















COUNTER VARIABLE | INITIAL VALUE 
cl! : eure "year' for the game 1960 
C2 M, market price 10 
C3 Cr number of fish caught 0 
C4 Me value of cash on hand 500 
C5 | Sr reproduction and survival factor 100 
er. sl ommberjon cian ince | S=SCSC~CSC*CS*~<S 300 
a | dummy storage for intermediate results 
C9 profit for year 0 
| tenter ct emmtoyes a 
Cll Po value of fishery equipment 150 
C12 Ny number of new employees at time T 0 
C13 AL. value of addition or maintenance to plant or 
; equipment at time T 0 
C14 | H,. average salary 30 
OE eS ee Oe 
C16 | x, amount spent on fish ladders at time T 0 
C17 De value of anti-pollution measures 150 
C18 | Wp | amount spent on anti-pollution measures at time 0 
C19 ) | stored value of V, tor year lL, Cl = 1960 
ee ee ee ee ee 
C21 stored value of Vy fOr yeatwo. .un Cl =1962 
C22 | stored value of Ve for year 4 , Cl = 1963 
C23 stored value of Ve for year 5 , Cl = 1964 
C24 stored value of Vp for year 6 , Cl = 1965 
O27 | | stored galuesct, Vy £05 Joes eG oko ae 
C26 stored value of Ve forivear( 57.5 1Cl =s19G6/ 
C27 stored value of Vp for year.9. ., Cli= 1968 
C28 stored value of Vy for year 10 , Cl = 1969 
C29 average value of counters 19 to 28. (C29 also contains the 


model number at the time of sign-on). 
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COURSE DOCUMENTATION 


Course Title: Matching Familiar Figures 
Grade: Grades 1 - 12 

Author: J. Dale Burnett 

Computer: IBM 1500 System 

Language: Coursewriter I1 

Date: October, 1970 


Division of Educational Research Services 
The University of Alberta 


Edmonton, Canada 
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Information for the Educator 


dbs 


Objectives and Purposes 


The program provides a computer-based presentation of Jerome Kagan's 
Matching Familiar Figures (MFF) test. This test may be used to 
provide measures on an "Impulsivity - Reflectivity" scale. This 
version of the test requires a student to select a matching figure 
from among six choices. 


Course Content 
The test contains two practise trials: 


Trials 1 
Triage 2 


cups 
rulers 


Following are twelve trials, presented in the order: 


Trial 3 : houses 
Trfaiyra % scissors 
Triaies : phones 
Trial 6 Sie.bears 
Leva? : trees 
Tria lS : leaves 
EYvial 9 S.cats 
Leial: LO : dresses 
Ere te i. : animals 
Pri alek?2 : lamps 
Terals13 : boats 
Trial ehé : cowboys 


For each trial the student is shown a picture of one figure separated 
from a set of six figures by a solid line. The student is required to 
select the figure from the set of six that is identical to the single 
figure. 


If a student selects an incorrect choice during the two practise trials 
a message is displayed giving the reason why he was wrong and he is 
branched back to the same problem and asked to try again. If he is 
correct he receives confirmation and is branched to the next problem 
in the sequence. The same pattern is repeated for the next twelve 
trials except if he is incorrect he is merely told to try again - no 
message is given explaining why he was wrong. 
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Age and Educational Level 


A non computer-based version of this test has been used for children 
in Grades 1 - 6. However, an upper limit has not been determined. 
It is desirable that the test represent a balance between not being 
too easy and not being too difficult. It is probable that this test 
would be satisfactory for children in grades 7 - 12 as well but this 
has not been validated. An adult version of the test exists that 
has eight choices instead of six for each item. The instructions 
may be given either visually on the CRT screen or verbally via ear- 
phones. If the instructions are presented on the CRT, it is obvious 
that the child must be able to read. A rough criteria might be to 
use the audio system for grades 1 - 6 and the visual system for 
grades 7 and over. 


Source of Course Content 


Kagan, J., Pearson, L., and Welch, L. Conceptual impulsivity and 
inductive reasoning. Child Development, 1966, 37, 583-94. 


Gupta, P.K. Correlates of reflection-impulsivity. Ph.D. Thesis, 
1970, Dept. of Educational Psychology, University of Alberta. 


Program Duration 


The program will take from 5 to 20 minutes depending on the student. 


Method of Instruction 


See number 2, Course Content, of this documentation. 


Audio Visual Requirements 


a. © Film reel UA122 is required. 
b. Special note on registration of students: If the instructions are 


to be given via the audio unit, the "student name" field must contain 


the integer "1". In this case audio tape AC-42 is required. 


If the instructions are to be presented on the CRT, no audio tapes 
are required. In this case the "student name" field must contain 
the integer "0". It is important that the operators are informed 
of the value to be given to the "student name" field, or the 
program will not execute properly. 


8. Response Modes 


The student uses the light pen to indicate his choice for each item. 
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Student Operating Requirements 


None. 

Teacher Supervision 
Not required. 
Evaluative Information 


Standard performance recordings are maintained. The following 
data is stored for each record: 


Student identifier 
Date 

Response identifier 
Match identifier 
Latency. 


No data has been analyzed at this time. This program will be 
used in the programmer's thesis. 


Auxiliary Materials 

None. 

Student Action 

Not available at time of documentation. 
Programmer and Author 

J. Dale Burnett. 

Availability 


Restricted distribution, communicate with author. 
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Intormmatzon! fon the Operator 


Fe 


Course Name and Segments 


Course Name: f4gut Segment 000. 


Dictionaries and Graphic Sets 


Standard Dictionary only. 
Functions Called 

en 

Macros Called 


dua007, 4tla007. 


Film Reels 


Film Reel: UA122, Frames 241-254 Inclusive. 


Audio Tapes 


Optional - see 14. Special Instructions. 
If used, audio tape AC-42 is required. 


Execution Time 


5 - 20 minutes. 


Response Times 


Response times are included with ep instructions. 
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Pre-Course Instruction Requirements 


None. 


Student Sign-On Command 


Standard Sign-on procedure. There are no code words. 


Teacher Supervision 


No required 


PrOeceor Messages 


None. 


Performance Recordings 


A performance pack or tape is required. The following data 
should be stored for each record: 


Student identifier 
Date 

Response identifier 
Match identifier 
Latency. 


Special Instructions 


Important note on registration of students. The "student name" 
field must contain either the integer "0" or the integer "1". 
An "0" indicates no audio tapes are used. 

An."1" indicates that audio tape AC-42 must be used. 
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Information for the Programmer 


aS 


4, 


Course Listings and Flow-Chart 


See documentation package in DERS library. 

The student is given two practise trials (picture of cups and 
picture of rulers) where he is required to choose the matching 
figure from among six choices. If he is wrong a message is given 
explaining the reason why he was wrong and he is branched back to 
the same problem and asked to try again. If he is correct he 
received the next problem in the sequence. The same pattern is 
repeated for the next twelve trials except if he is incorrect he 
is merely told to try again - no message is given explaining why 
he was wrong. Summarizing, the program consists of a linear 
sequence of fourteen trials where the student must correctly 
answer the present trial before going on to the next trial. 


Macros Called 


Macro duado7 - press space bar to continue 
tha007 - time out 


See The University of Alberta 1500 Instructional System - Macro 
Guide for a complete description of these two macros. 


Buffers Used 

After the student signs-on, buffer 0 will contain the "student 
name" field that is specified at the time of registration. This 
field must contain either an "0" or an "1". 

Buffer O will also contain the student's response to an 2p 
instruction. This is a standard feature of the 1500 system. 
Counters Used 

Cl - contains the value of the integer extracted from buffer O at 


the time of sign-on. O- no audio 
1- use audio 
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Switches Used 


Sl - set equal to 1 if a value of 1 is extracted from buffer O at 
the time of sign-on. 


Return Registers Used 


None. 


Functions Called 


eh 


Graphic Sets and Dictionaries 


None. 


Special Programming Features 


The value of buffer O after the student signs-on is searched using 
function et (extract integer) and the numerical value is stored in 
counter 1. This value must be either 0 or 1. If the resulting 
counter 1 is equal to 1, the logic branches to that portion of the 
program that uses audio for all instructions, otherwise no audio is 
used. 


This procedure makes use of the fact that the value of the "student 


name" field at the time the student number is registered is stored 
in buffer O when the sign-on procedure is completed. 


Programmers Name 


J. Dale Burnett 
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Symbolic Name 





Group Mark # 1 


Fil 


cul 


CU2 


cu3 


Cu4 


Q-Tone 


GM 


BOM 


EOM 


BOM 


EOM 
BOM 
EOM 
BOM 
EOM 


BOM 


EOM 


| 
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Message 


Matching Familiar Figures 

You will be shown a picture of something you know 
and beside it will be some pictures that look like 
it. There will be a line between the picture that 
is by itself and the others. You will have to 
choose the picture that is just like the picture 
which is by itself. Now look at the screen above 
the keyboard. Use the light pen to point to the 
square on the screen that is in the same place as 


the picture you choose. Let's do some for practise. 


Please point to the square that is in the same 


place as the picture you choose. 
Please answer as quickly as possible. 
Very good. That was the correct choice. 


No. You pointed to the square on the left in the 
top row. This means you chose the cup with the 

dots on it. The picture by itself has lines on it. 
You must choose the cup that is the same as the 


one which is by itself. Please try again. 
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Symbolic Name 


CUS 


CUu6 


CU7 


Cus 


cu9 


A | at Se 
pn 


EOM 


BOM 
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BOM 


EOM 


BOM 


EOM 
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Message 





No. You pointed to the middle square in the top 
row. This means you chose the plain cup with the 
square handle. The picture by itself has lines on 
it. You must choose the cup that is the same as 


the one which is by itself. Please try again. 


No. You pointed to the square on the right in the 
top row. This means you chose the striped cup 
with no handle. The picture by itself has a 
handle. You must choose the cup that is the same 


as the one which is by itself. Please try again. 


No. You pointed to the middle square in the bottom 
row. This means you chose the cup with the triangle: 
On it. The picture by itself has lines on it.. You 
must choose the cup that is the same as the one 


which is by itself. Please try again. 


No. You pointed to the square on the Bey: in the 
bottom row. This means you chose the blank cup 
with the curved handle. The picture by itself has 
lines on it. You must choose thesedp that is the 
same as the one which is by itself. Please try 


again. 


You must point to one of the six squares on the 


screen. Please try again. 
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Symbolic Name 





Group Mark # 2 


RU1 


RU2 


RU3 


RU4 


RU5 


RU6 


Q-Tone 


GM 


BOM — 


EOM 
BOM 
EOM 


BOM 


KOM 


BOM 


BOM 


EOM 
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Message 


Please point to the square that is in the same 


place as the picture you choose. 
Please answer as quickly as possible. 


Very good. That was the correct choice. Now we 
are going to do some that are a little bit harder. 
Find the picture that is just like the one that is 


by itself. 


No. You pointed to the square on the left in the 
top row. This means you chose the short ruler 
with marks on it. The ruler by itself is a longer 
ruler. Choose the ruler that ‘ the same as the 


one which is by itself. Please try again. 


No. You pointed to the middle square in the top 

row. This means you chose the shorter ruler with 
no marks on it. The ruler by itself is a longer 
ruler. Choose the ruler that " the same as the 


one which is by itself. Please try again. 


No. You pointed to the square on the right in the. 
top row. This means you chose the short ruler 


with no marks on it. The ruler by itself is a 
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6 ee 





longer ruler. Choose the ruler that is the same 


as the one which is by itself. Please try again. 


No. You pointed to the square on the left in the 





bottom row. This means you chose a shorter ruler. 
The ruler by itself is a longer ruler. Choose the 
ruler that is the same as the one which is by 


itself. Please try again. 


picture by itself. Please try again. 


EOM 
RU8 BOM No. you pointed to the middle square in the bottom 
row. This means you chose a shorter ruler with 
| marks on it. The ruler by itself is a longer 
: ruler. Choose the ruler that is the same as the 
one which is by itself. Please try again. 
EOM 
RUQ BOM You must point to one of the six squares on the 
| screen. Please try again. 
! EOM 
GEl | BOM Please point to the square that is in the same 
place as the picture you choose. 
| EOM 
GE2 BOM Please answer as quickly as possible. 
EOM 
GE3 BOM Very good. That was the correct choice. 
EOM 
GE4 BOM No. Pick the picture that is the same as the 
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GE5 You must point to one of the six squares on the 
screen. Please try again. 
EOM 
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APPENDIX D 


HANDOUT ON 


STUDENT USE OF THE COMPUTER TERMINALS 
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STUDENT USE OF THE COMPUTER TERMINALS 


Notice that each terminal contains 
1. a cathode-ray display screen (like a small TV screen) 
2. a keyboard (like a typewriter) 


3. a light pen (located on the right-hand side of the 
display screen) 


4. An slide projector (this is the large ‘box! located to 
left of the display screen) 


5. a tape recorder (this is the large 'box' located to the 
right of the display screen). 


| Sam cathode-ray display screen 


keyboard 


All of the instructions are shown on the cathode-ray display screen. 
Some of the courses may ask you to use either the light pen or the 
keyboard, or both. However you do not have to worry about what you are 
supposed to do, since instruction will always be given on the display 
screen explaining what is wanted. 






Se 


tape recorder 






A course may use the slide projector to show you some pictures 


or the tape recorder to play a message, but most courses only use the 


display screen, light-pen, and the keyboard. 
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Using the keyboard 


Most courses will ask you to type an answer to a question that appears 
on the display screen, Thus it is important that you know how to use the 


keyboard, 


Step l 


Step 2 


Step 3 


Whenever the computer wants you to type an answer, the letter K 
(the letter K means keyboard response) will appear in the bottom 
right corner of the display screen, 


Never type in an answer unless there is a K in the bottom right 
corner of the display screen, 


Always type in an answer when there is a K in the bottom right 
corner of the screen, 


Press the keys on the keyboard to give your answer, Usually 
your answer will appear on the display screen. If you make 

a mistake before you are finished hold the "altn coding" button 
on the top left corner down and then press the backspace button 
(on the top right corner of the keyboard) until your answer is 
erased from the display screen, Then retype your answer. 


This is the final step. You must let the computer know that 
you have finished your answer, This is done by "entering" 

your answer, To enter an answer first press the "altn coding" 
button on the top left corner down and then press the space bar, 
Be sure to hold the "altn coding" button down while you press 
the space bar. After you have entered your answer, the letter 
K will disappear from the bottom right corner of the display 
screen, 





Remember, if you have finished typing your answer and the letter 
K is still in the bottom right corner of the display screen, 
you must "enter' your answer. 


Using the light pen 


Remember that the light pen is found on the right-hand side of the display 


screen, 


Many of the courses will ask you to indicate your answer by pointing your 
light-pen at a particular point (usually called a ‘target') on the display 


screen, 
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Step 1 Whenever the computer wants you to use the light pen, 
the letter P (the letter P means light-pen response) 


will appear in the bottom right corner of the display 
screen. 


Step 2. Point the light pen at the 'target' (this is usually 
a square) and press it (firmly, but not too hard) 
against the screen. As soon as the computer recognizes 
what you have pointed to, the letter P will disappear 
from the bottom right corner of the display screen. 


-REMEMBER 
1. The course will contain complete instructions on what is required. 


2. The letter K in the bottom right corner of the display screen 
means that you are to type an answer. 


3. The letter P in the bottom right corner of the display screen 
means that you are to use the light pen. 


4. If there is no K or P in the bottom right corner, you are 
not expected to give an answer. Be sure to wait for the 
K or P to appear before answering a question. 


5. If you are not sure of what you are supposed to do, please 
raise your hand and the computer operator will help you. 


The first time you use the terminals everything is new and appears 
strange, but it is much like learning how to use a new-tool - once 
you have used it a few times, you become familiar with the way it 
works and it becomes much easier to use. 
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FORM REQUESTING PARENT'S PERMISSION 
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Wiad er Rae MAGS 


January 5, 1971 178 


To the Parents of : 
Sa acl en A a a me ei a 
Your student has been selected by a random procedure for participation 


in a study tc be conducted at the University of Alberta, 


This study will provide further data on two relatively new developments 
in Education - that of computer assisted instruction and that of simulation 
gaming. Each student will be transported to the Education building on the 
- University of Alberta campus where he/she will participate in a simulation 
game based on the salmon fishing industry in British Columbia. This will 
take approximately one hour, All of the selected students will also receive | 
a set of tests which will be given at Wellington Junior High School following 
the simulation game, There should be a minimum of disruption from the normal 
school schedule, All results will be kept completely anonymous and confidential 


- only results of the group as a whole will be reported, 


The study has the endorsement of the Edmonton Public School Board, and the 
principal of Wellington Junior High School, Mr. J. Marles, 


It would be appreciated if this form could be Signed and returned as soon 


as possible, 
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RAW SCORES ON 
MARKER TESTS FOR COGNITIVE FACTORS 
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APPENDIX F 


RAW SCORES ON MARKER TESTS FOR COGNITIVE FACTORS 


Pn ee 








Subject 
THE TLI@ oleae Macd, Maedo Nl N-3. R-1) R-4) V-lo V-2, MFEGY MFF-2 


1 18 44 6 8 47 59 12 10 12 8 2) 25> 
y’ 22 138 9 14 ae 4l Fa! 18 20 V2 2 445 
3 Li 48 10 11 30 41 6 rE Ly 4 22 320 
ee LS 55 9 10 18 Pal 6 12 6 7 Ge 222 
a 2 94 15 15 20 36 6 13 10 3 O 548 
be el3 59 cL 6 24 30 4 9 12 us 2 668 
7 0 79 4 16 23 ao 7 8 8 8 5 85 
8 14 70 LS 20 22 40 3 9 11 5 16a 20% 
9 18 oo 11 i? ya 22 5 re 20 9 O 546 
CD oeerat WE 7 13 14 oy) 63 3 y, 15 iS ae 258) 
11 13 40 4 9 20 59 8 u 2 8 he 
12 3 50 3 8 ao 40 ) S 6 10 17 196 
13 15 55 9 a 38 50 4 9 EL 8 1 534 
Faw, ©2353 110 Ve, 19 47 67 Le 13 23 13 OF 42% 
16 15 SS zs) 6 38 58 0 5 3 i 5 134 
WT) geese: 119 MS 17 58 85 22 22 18 Like! 5 005 
18 20 61 12 16 22 Se 4 9 8 14 2) «6.328 
ro Zt 2 17 is 42 54 BT yi 1k 10 i 632g 
20 «14 62 20 20 33 32 A 9 12 5 11 138 
21 20 47 4 8 27 40 5 10 10 ll OF -< 252 
2h 22. 45 2 zg 33 38 AONE 8 EZ 5 Lee og 
23 ¥2 69 3 5 8 21 6 12 15 18 O 583 
26 16 49 al 20 23 33 f 10 19 10 11 126 
27 9 36 di re 25 26 5 12 4 5 2 32 
28 14 43 24 21 34 49 10 18 14 "i 1 438 
3h 14 70 20 13 of 56 4 12 8 9 3. 2552 
Se ae Je: 25 18 31 a2 8 10 it 8 1 617 
Se ko 61 26 16 38 46 6 15 10 12 15 274 
34 11 46 9 20 23 30 7 9 Ls if Lees 
fo) 13 0 30 30 35 45 7; 17 26 19 1 eee 
So = 25 og. 6 9 41 52 4 10 2 8 4 319 
2 16 61 8 13 32 42 7 10 3 5 O 528 
382-17 26 15 Ue 37 55 7 iN 9 iv fe 34 
a LG 102 12 va 29 34 2 LZ 22 18 3. dt 
40 15 td 10 13 24 48 g is 4 2 1 eG 
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APPENDIX F (Continued) 


Subject 
EDG iL 1-3 Ma-2 Ma-3 N-l N-3 R-1 R-4 V-1 V-2 MFF-1 MFF-2 


41 0 3 fs 29 Le 28 43 5 9 ie 9 >) 290 
AZ ee 131 10 14 33 47 8 14 18 15 Lees) 
43 1 68 20 Bi 23 23 8 rhs 8 poh 13 192 
45 LS 84 10 16 a3 32 1 6 14 8 6 181 
46 18 67 5 4 ag oe. 5 3 14 13 Dea 
47 26 7.3" 23 18 30 36 14 20 23 20 a2) SLs 
48 5 46 & iL 18 19 3 8 4 2 10) > 233i 
49 20 34 25 14 38 ys ) 11 d 4 a gece 
oil 20 90 16 19 25 eo 8 i2 10 fi lg ea 72 
Vs = 78's 76 23 19 30 33 8 10 15 8 oh Bee 
Sie pes 62 ED 25 27 32 vi 16 28 23 O 434 
54 7 91 9 ei 32 43 5 10 6 9 4 274 
Clay bards 49 8 14 30 38) ua 13 ile 7 1 612 
TE al 89 24 28 36 45 18 Se 9 if OF 7a. 
og} A) Ey 2 22 42 ye 1 La nS 11 LeU. 
58 18 64 16 26 43 oY] 8 3 8 3 Saees le 
59 16 68 5 9 42 48 7 12 15 9 Sige ee 
60721 2 20 23 a2 oo 9 16 32 30 O 416 
61 14 58 y; ll 22 28 7 10 10 9 Tipe Pee) 
62 Vey 40 10 Ve 30 SW) Le 1} 20 8) i 601 
eh a 30 a 20 60 74 8 15 Ly 5 Die 202 
64 19 37 12 14 22 21 1 9 18 6 10 209 
65 i? 66 11 19 ab 44 2 a 8 3 07 7305 
66 14 74 is 25 42 38 10 9 10 8 O 643 
67 19 fp 29 rs 42 63 6 8 14 2 182 
68 14 re. 21 18 45 55 10 LZ 1 3 167 Bos 
69 7 ae 23 22 29 a5 9 12 16 EE 6 189 
y {0 40) 65 i 18 28 31 1 10 10 2 2 ee 
72 11 83 8 14 al 36 7 1B) Ly. 16 4. 4415 
re: 19 G7 13 21 28 De) 6 BI 26 16 VRE HF 
7A 24 120 L Ea 29 28 13 15 19 15 2 408 
i is5 ea) 68 17 16 38 63 9 16 8 10 Oy {08 
Nagao, 


Note: MFF-1: Number of errors on MFF test 


MFF-2: Average latency time per figure on MFF test (tenths of 
seconds) 
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